Paperonium

← Back to the plain-language case

Abstract reports 58 (53%) of 113 resistant sites; Results' own counts give 57 (51%) of 112

Lancet Infect Dis (2014) Β· doi:10.1016/S1473-3099(14)70855-2 Β· PMC4178238

Summary

This systematic review reports its headline result in the abstract as chloroquine resistance in "58 (53%) of 113 assessable study sites". The Results instead give 57 (51%) of 112 site estimates, and the body's own sub-counts support that figure: 177 site estimates minus 65 that could not be categorised (37 + 15 + 4 + 4 + 5) leaves 112, and resistance categories 1, 2 and 3 (35 + 8 + 14) add up to 57. The abstract figure is also wrong on its own terms, because 58/113 = 51.3%, which rounds to 51%, not 53%. The qualitative finding, that about half of assessable sites showed resistance, is unchanged. But the abstract's one headline prevalence figure, the number most likely to be quoted, is about 2 percentage points too high, and its count and denominator match nothing in the paper.

What the paper reports

Abstract, Findings:

We identified 129 eligible clinical trials involving 21β€ˆ694 patients at 179 study sites and 26 case reports describing 54 patients. Chloroquine resistance was present in 58 (53%) of 113 assessable study sites, spread across most countries that are endemic for P vivax.

Results, fifth paragraph:

Estimates of chloroquine efficacy could be derived for 177 sites from 144 treatment groups. In 65 (37%) of these sites chloroquine resistance could not be categorised for the following reasons: early administration of primaquine (37 sites); no documentation of parasitological data at day 28 (15 sites); sample size of fewer than ten patients (four sites); good response documented in asymptomatic patients (four sites); and administration of only a single dose of chloroquine (five sites). Overall P vivax was defined as chloroquine resistant for 57 (51%) of 112 site estimates, 35 (61%) of these sites fulfilled the predefined criteria for category 1 resistance, eight (14%) for category 2 resistance, and 14 (25%) for category 3 resistance.

Table 1, "Site estimates of chloroquine efficacy" column, "Overall" row:

Overall | 129 | 230 | 177

Discussion, ninth paragraph:

Our definitions were conservative, with data excluded from almost 40% of studies that we thought could be confounded. Almost half of the informative studies suggested evidence of reduced chloroquine efficacy.

What the paper's own numbers show

Method. I checked each printed percentage against its own fraction. I rebuilt the assessable denominator from the Table 1 site total and the five listed reasons for exclusion, and rebuilt the resistant count from the three resistance categories. I checked the Table 1 site total by each of its three breakdowns (location, year, origin of patients). Finally, I searched for fractions that would round to 53%. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Resistant sites, abstract Abstract, Findings 58 (53%) of 113 58/113 = 51.3%, which rounds to 51%, not 53%
Resistant sites, Results Results, para 5 57 (51%) of 112 57/112 = 50.9% (51%); consistent
Chloroquine site estimates Table 1; Results, para 5 177 134 + 13 + 22 + 8 = 177 by location; 8 + 41 + 80 + 48 = 177 by year; 22 + 21 + 134 = 177 by origin
Uncategorisable sites Results, para 5 65 (37%) 37 + 15 + 4 + 4 + 5 = 65; 65/177 = 36.7% (37%)
Assessable sites Results 112; Abstract 113 112 / 113 177 βˆ’ 65 = 112
Resistant sites = categories 1 + 2 + 3 Results 57; Abstract 58 57 / 58 35 + 8 + 14 = 57
Category shares Results, para 5 61% / 14% / 25% of 57: 61.4% / 14.0% / 24.6% (match); of 58: 60.3% / 13.8% / 24.1% (35/58 would print as 60%, not 61%)
Implied sensitive sites derived β€” 112 βˆ’ 57 = 55 (Results); 113 βˆ’ 58 = 55 (Abstract)
Fractions that round to 53% derived 53% 58/109, 58/110, 60/113 or 59/112; none of these counts appears in the paper
Size of the discrepancy derived β€” printed 53% is 1.67 points above 58/113 and 2.11 points above 57/112; the two fractions differ by 0.43 points

Figure

Abstract and Results give different resistant-site counts, and the abstract percentage does not match its own fraction

Figure 1. The two passages that report the overall prevalence of chloroquine-resistant P vivax sites. Values printed in the abstract are in orange; values printed in the Results, which agree with the body's sub-counts, and values recomputed from the paper's numbers are in blue. The abstract's count (58) and denominator (113) are each one site higher than the Results' (57 and 112), and its percentage (53%) does not follow from its own fraction (58/113 = 51.3%). The lower panel rebuilds the Results' figures from the Table 1 site total, the five listed reasons for exclusion and the three resistance categories. All numbers are from the paper or from recompute.py.txt.

Why it matters

The abstract's sentence is the paper's single summary figure for how widespread chloroquine-resistant P vivax is, and it is the version most likely to be quoted in later reviews, guideline background documents and resistance summaries. The error is small: the figure the body supports is 51% (57/112), and the abstract's own fraction also gives 51%. Its direction and the qualitative reading ("almost half of the informative studies", in the Discussion's words) are unchanged, and so are the conclusions that resistance is geographically widespread and that monitoring needs to improve. The practical consequences are that readers who quote the abstract carry a percentage about 2 points too high, and that the counts 58 and 113 cannot be reconciled with the paper's own tables and text. The abstract and the body disagree in three places at once (count, denominator and percentage), which will puzzle anyone who tries to check or update the analysis.

Explanations considered

  1. A different unit of analysis. The abstract says "assessable study sites" and the Results say "site estimates". The paper gives only two site totals: 179 study sites for all drugs (Abstract) and 177 site estimates of chloroquine efficacy (Results; Table 1, whose three breakdowns each sum to 177). The only assessable denominator the paper derives is 177 βˆ’ 65 = 112. No figure in the text, tables or legends gives 113. Applying the 65 exclusions to the 179 study sites would give 114, not 113. This does not account for the numbers.
  2. A different set of resistance categories. Resistance is defined as categories 1, 2 or 3, and these sum to 35 + 8 + 14 = 57. No further resistant subgroup is described that could add a 58th site. Both versions imply 55 sensitive sites (112 βˆ’ 57 = 55 and 113 βˆ’ 58 = 55). The abstract therefore has one extra site that is counted as resistant. The printed category shares also fit a base of 57 rather than 58 (35/58 = 60.3% would print as 60%, not 61%). This does not account for the numbers.
  3. Rounding convention. 58/113 = 51.33%. No rounding rule turns that into 53%. A printed 53% would need 58/109, 58/110 or 60/113 (or 59/112), and none of these counts appears in the paper. This does not account for the percentage.
  4. A conversion artefact in the archived text. The same sentences appear in the Lancet's own typeset PDF (the online-first version), in the PMC HTML and in the Europe PMC XML. This is not a conversion artefact.
  5. A data update between versions, such as an earlier count left in the abstract. The body is internally consistent throughout (65 = 37 + 15 + 4 + 4 + 5; 65/177 = 37%; 57/112 = 51%; the category shares of 57). That points to the abstract's figure as the inconsistent one, but the paper gives no information on how it arose, and even such a history would not explain why 53% does not equal 58/113.
  6. The supplement. The single supplementary appendix contains reference lists and a reference table only. It has no site-level categories or counts, so it cannot reconcile 58/113 with 57/112.

Status

Suggested correction

  1. In the abstract's Findings, replace "Chloroquine resistance was present in 58 (53%) of 113 assessable study sites" with "Chloroquine resistance was present in 57 (51%) of 112 assessable study sites". This matches the Results, the five listed exclusions from the 177 site estimates in Table 1, and the 35 + 8 + 14 resistance categories.
  2. If the authors' dataset shows that 58 of 113 is the correct count, then the Results paragraph (the 112 denominator, the 65 exclusions and their breakdown, the 57 total and the category counts and shares) needs correcting instead. The abstract percentage would then still need to change from 53% to 51%, because 58/113 = 51.3%.