← Back to the plain-language case
The paper calls store sales primary; the registered primary, women's diet quality, is reported as secondary
PLoS Med (2021) · doi:10.1371/journal.pmed.1003729 · PMC8423266 · NCT03518151
Summary
ClinicalTrials.gov NCT03518151 (WRAPPED1) has a single version, submitted on 2018-04-25 (658 days after the registered study start and 360 days after primary completion), which names one primary outcome, women's dietary quality score from a 20-item food frequency questionnaire (FFQ) at 3 months, and lists weekly store sales of fresh fruit and vegetables and of confectionery over 12 weeks as secondary outcomes. The published primary report reverses that order: it names "store sales (primary outcome)", calls women's and children's diets "secondary outcomes", and reports every outcome at 3 and 6 months. The paper cites the registration three times and never mentions that its outcome order differs from the registered one. The registered primary outcome is reported in Table 2 (+0.29 SD, P = 0.04, adjusted for the Index of Multiple Deprivation, IMD; +0.26 SD, P = 0.06, fully adjusted), but the abstract gives no numbers for it.
What was registered
The registry record history lists a single version. It was retrieved through the ClinicalTrials.gov history endpoint (/api/int/studies/NCT03518151/history/0) and checked against the current public API v2 record, which matches it.
| Registry version | Date | Primary outcome | Secondary outcomes |
|---|---|---|---|
| v0 (the only version) | submitted 2018-04-25; first posted 2018-05-08; last update posted 2018-05-08 | Dietary quality score, 3-month follow-up | Child's dietary quality score (3 mo); women's daily fruit and vegetable portions (3 mo); women's weekly confectionery intake (3 mo); weekly sales of fresh fruit and vegetables (12 wk); weekly sales of confectionery (12 wk) |
The registered primary outcome reads, verbatim:
Dietary quality score — "The primary outcome is women's dietary quality measured using a 20-item food frequency questionnaire (FFQ) to create a standardised dietary quality score (validated against 100-item FFQ and red cell folate)." Time frame: "3 month follow-up post intervention commencement"
The two registered store-sales outcomes read:
Weekly sales of fresh fruit and vegetables (secondary) — "Total weekly sales of fresh fruit and vegetables collected via electronic sales data". Time frame: "12 week period following intervention commencement"
Weekly sales of confectionery (secondary) — "Total weekly sales of confectionery collected via electronic sales data". Time frame: "12 week period following intervention commencement"
Registry dates (all marked "actual"): study start 2016-07-06; primary completion 2017-04-30; study completion 2017-11-30. Enrollment: 150 (actual). Because v0 is the only version, the registry primary outcome was never changed. The record was created after data collection for the primary outcome had ended. The paper agrees: it says the study "took place between April 2016 and March 2017" and that "Recruitment occurred in 3 waves between July and September 2016."
What the paper reports
The paper names store sales as its primary outcome in two places:
It aims to assess whether creating a healthier layout in discount supermarkets in England improves the healthiness of store sales (primary outcome) and the purchasing and dietary behaviours of women customers aged 18 to 45 years (secondary outcomes) after 3 and 6 months.
— Introduction, final paragraph
The data collected included 9 months continuous store sale transaction data (primary outcome) and participant loyalty card data (secondary outcome) provided by the collaborating supermarket to cover 3 time periods
In order to obtain an understanding of intervention effects on household members’ diets (secondary outcomes), interview-administered telephone questionnaires obtained information about participants’ diets and diets of their children aged 2 to 6 years (where applicable) at 3 time points: baseline (time 1) and 3 (time 2) and 6 months (time 3) following intervention commencement.
— Section 2.4, Outcome measures
The store-sales analysis covers four product groups: "Weekly sales per store of fresh fruits and vegetables, frozen vegetables, confectionery, and intervention checkout items" (Section 2.6). The abstract's quantitative results are store-sales and loyalty-card purchasing results. Its first concluding sentence is "Healthier supermarket layouts can improve the nutrition profile of store sales and likely improve household purchasing and dietary quality."
The paper mentions the registration in three places, each time without comment on its content:
The trial registration number is NCT03518151.
— Abstract, Methods and findings
ClinicalTrials.gov NCT03518151 (pre-results)
— Abstract, "Trial registration" line
it was registered with ClinicalTrials.gov (NCT03518151, pre-results).
— Section 2.1, Study design and setting
The paper describes itself as "a pilot study with a prospective matched controlled cluster design" (Section 2.1). The article text (excluding the reference list) contains none of the strings "retrospectiv", "amend", "changed", "protocol", "analysis plan", "prespecified" or "pre-specified". The word "deviat" occurs only in "standard deviation". The one statement about decisions made after data collection concerns analysis methods, not which outcome is primary:
The distributions of the data were unknown in advance of study commencement because this was a pilot study. The research team therefore made decisions about the statistical analyses following (i) completion of initial analyses to determine the shape of the outcome data; and (ii) consultation with the literature to identify appropriate statistical methods for the various outcome variables.
— Section 2.6, Statistical analysis
The discrepancy
| Outcome | Registered (v0, 2018-04-25) | In the paper | Status |
|---|---|---|---|
| Women's dietary quality score (20-item FFQ, standardised) | Primary, 3-month follow-up | Secondary ("household members’ diets (secondary outcomes)"); reported at 3 and 6 months (Results 3.4, Table 2) | demoted; time point changed (6 months added) |
| Weekly sales of fresh fruit and vegetables | Secondary, 12-week period after intervention start | Primary ("store sales (primary outcome)"); interrupted time series at intervention, 3 and 6 months (Results 3.2) | added as primary; time point changed |
| Weekly sales of confectionery | Secondary, 12-week period after intervention start | Primary (same); at intervention, 3 and 6 months | added as primary; time point changed |
| Frozen-vegetable sales; intervention checkout-item sales | Not registered | Part of the store-sales (primary) analysis (Sections 2.6, 3.2) | added |
| Child's dietary quality score | Secondary, 3 months | Secondary, 3 and 6 months (Table 2) | matched; time point changed (6 months added) |
| Women's daily fruit and vegetable portions | Secondary, 3 months | Secondary, 3 and 6 months (Table 2) | matched; time point changed (6 months added) |
| Women's weekly confectionery intake | Secondary, 3 months | Secondary, 3 and 6 months (Table 2) | matched; time point changed (6 months added) |
| Loyalty-card purchasing (fruit and vegetables, confectionery, other categories) | Not registered | "participant loyalty card data (secondary outcome)" (Sections 2.4, 3.3) | added |
| Economic outcomes (grocery spend, total store sales) | Not registered | Reported (Section 3.5) | added |
Every registered outcome is reported, so nothing is missing. The difference is in the outcome hierarchy. The only registered primary outcome is presented as secondary. Two registered secondary outcomes, together with two unregistered product groups, become the primary outcome, and a 6-month time point is added throughout.
Figures
Figure 1. Trial timeline. Blue items are registry dates; orange items come from the paper or the publisher's article page. The only registry version (v0) was submitted on 2018-04-25. That is 658 days after the registered study start (2016-07-06) and 360 days after the registered primary completion (2017-04-30). The orange bars show the study period (April 2016 to March 2017) and the recruitment window (July to September 2016) as the paper reports them. No later registry version exists, so the primary outcome was never changed in the registry. The paper (received 2020-10-15, accepted 2021-07-12, published 2021-09-07) appeared while v0 was still the only version.
Figure 2. The registered outcome hierarchy (left, ClinicalTrials.gov v0, 2018-04-25) compared with the hierarchy in the paper (right). Each connector is labelled with what changed. The sole registered primary outcome, women's dietary quality score at 3 months, is reported as a secondary outcome (DEMOTED), and a 6-month time point is added. The registered secondary weekly sales outcomes become the paper's primary outcome (PROMOTED, in the table above "added as primary"), and the time frame changes from 12 weeks to 3 and 6 months. Loyalty-card purchasing, frozen-vegetable and checkout-item sales, and economic outcomes appear only in the paper (ADDED). The bottom panel gives the paper's results for the registered primary outcome (Table 2), with the 3-month headline store-sales results for comparison, and notes that the paper discloses nothing about the difference.
Why it matters
A primary outcome named in advance limits the choices that can be made after the data are seen: which outcome is the main test, which time point counts, and what the abstract leads with. Here the only registration was created after primary completion. The paper's hierarchy then differs from that registration and the paper does not say so. A reader who opens the cited NCT number finds a different primary outcome from the one the paper names, and the paper gives no explanation.
The two hierarchies lead to different headlines. The abstract's numerical results and its first conclusion are about store sales. Fruit and vegetable sales rose by +1.71 SD (95% CI 0.45, 2.96; P = 0.01) at 3 months and +2.42 SD (0.22, 4.62; P = 0.03) at 6 months. Confectionery sales fell by −1.05 SD (−1.98, −0.12; P = 0.03) at 3 months and −1.37 SD (−2.95, 0.22; P = 0.09) at 6 months (Abstract; Results 3.2). The registered primary outcome is reported in Results 3.4 and Table 2:
| Women's diet quality score, intervention vs control, change from baseline | Beta (SD) | 95% CI | P | n |
|---|---|---|---|---|
| 3 months, adjusted for IMD (registered time point) | 0.29 | 0.01, 0.57 | 0.04 | 138 |
| 3 months, adjusted for IMD and DAG-selected confounders | 0.26 | −0.01, 0.54 | 0.06 | 132 |
| 6 months, adjusted for IMD (not a registered time point) | 0.10 | −0.16, 0.36 | 0.45 | 125 |
| 6 months, fully adjusted | 0.10 | −0.17, 0.37 | 0.48 | 119 |
Table 2 labels the IMD-only model "Unadjusted"; its footnote says "* Unadjusted = adjusted for IMD (to control for similarities between pairs of stores)." The Results text reports the 3-month IMD-adjusted estimate first: "After adjustment for IMD, diet quality score at 3 months follow-up increased from baseline by 0.29 SDs ((95% CI 0.01, 0.57), P = 0.04) among intervention women compared to those shopping in control stores."
So the registered primary outcome points the same way as the headline sales results. Its 95% CI excludes zero only in the IMD-adjusted model, and the 6-month estimate is close to zero. The abstract summarises it only as "Changes in dietary variables were predominantly in the expected direction for health benefit." The Discussion calls the diet findings "more equivocal". The paper also says it is "a pilot study, and, therefore, not powered to detect an effect" (Abstract). If the registered order had been kept, the abstract would have led with a borderline 3-month dietary effect rather than with store-sales effects. Readers and systematic reviewers should know which of these the trial was registered to test.
Explanations considered
- Paper role (secondary analysis or companion paper). This does not explain the difference. This is the main outcomes report of WRAPPED1: it reports store sales, loyalty-card purchasing and all four registered dietary outcomes, and its sample (n = 150; 62 intervention, 88 control) matches the registered enrollment of 150. It cites no other report of the registered primary outcome. Reference 59 (BMJ Open 2020;10:e036758) is the protocol of the separate WRAPPED natural-experiment study and is cited only under "Directions for future research".
- Disclosure in the paper. None found. The NCT number appears three times with no comment on its content. No passage compares the reported outcomes with the registered ones, and none says that registration happened after the study. The limitations paragraph covers the lack of randomisation, being "underpowered", and implementation that was outside the team's control, but not the outcome hierarchy. The statement in Section 2.6 about decisions made after initial analyses covers the choice of statistical methods, not the choice of primary outcome. The only supplement (S1 File) contains a flow diagram, DAGs and store-pair time series; it includes no protocol, statistical analysis plan or note on outcome changes.
- Protocol amendment. Not applicable in the registry, which has a single version that was never amended. The paper cites no protocol or statistical analysis plan for this pilot, so no protocol-level designation could be checked.
- Different wording for the same outcome. Ruled out. An individual FFQ-based dietary quality score and aggregate store sales from electronic transaction data are different measures. The registry itself lists both, the first as primary and the second as secondary.
- Pilot status. The paper calls the study a pilot, which might explain a flexible approach to analysis. However, the registration was created after data collection and still names one primary outcome. Pilot status does not explain why the paper names a different primary outcome without saying so.
- Time points. The paper explains its 0–3 and 3–6 month windows (Section 2.4: habit formation, sustained change), but never compares them with the registered 3-month and 12-week time frames. This adds a second undisclosed difference and does not explain the swap of primary and secondary outcomes.
There may be legitimate reasons for the paper's hierarchy, for example the store-level sales data being the most complete data source. This write-up does not assess them. The issue documented is only that the difference from the registered primary outcome is not disclosed.
Status
As of 2026-10-23:
- Registry history: one version only (v0, submitted 2018-04-25, posted 2018-05-08; last update submitted 2018-04-25 and posted 2018-05-08). Retrieved through the ClinicalTrials.gov history endpoints (
/api/int/studies/NCT03518151?history=true,/history/0) and the public API v2 record, which agree. - Publisher version: PLoS Med 18(9): e1003729, received 2020-10-15, accepted 2021-07-12, published 2021-09-07. The PMC archive copy (PMC8423266) was used for all quotations, and each quotation was checked verbatim with
src/show.py PMC8423266 --grep. The publisher's article page shows no correction, expression of concern or retraction; the only matches for those terms are links to the journal's policy pages. The PLOS peer-review history page could not be retrieved (HTTP 404). - Corrections and notices: none in Crossref (no update records), PMC or Europe PMC (
src/known.py). - PubPeer: not checked. Both the search page and the API returned HTTP 403 (bot challenge).
- Sources: ClinicalTrials.gov NCT03518151 (v2 API and record-history endpoints); PMC8423266 full text; journals.plos.org article page for doi:10.1371/journal.pmed.1003729; Crossref, PMC and Europe PMC notice searches. The figures are drawn by
make_figures.py.txtfrom the dates and outcomes listed in that file.
Suggested correction
A correction or clarification would make the outcome hierarchy match the registry, or explain why it does not. It could do three things:
- State in the Methods (Sections 2.1 and 2.4) and in the abstract's Trial registration line that the trial was registered on 2018-04-25, after data collection, and that the registration names women's dietary quality score (20-item FFQ) at 3 months as the sole primary outcome, with weekly store sales of fresh fruit and vegetables and of confectionery over 12 weeks as secondary outcomes.
- Either present the registered primary outcome as primary, or say that store sales were designated primary in the paper, when this was decided and why. Also state that the 6-month time point and the frozen-vegetable, checkout-item, loyalty-card and economic outcomes were not in the registration.
- Give the registered primary result in the abstract with its estimate, CI and P value: 3 months, +0.29 SD (0.01, 0.57), P = 0.04 adjusted for IMD; +0.26 SD (−0.01, 0.54), P = 0.06 fully adjusted.