Paperonium

← Back to the plain-language case

Abstract gives the 14-week odds ratio as 2.51; Table 2 counts give 5.02

PLoS Med (2019) Β· doi:10.1371/journal.pmed.1002832 Β· PMC6588214

Summary

The abstract of this randomized trial reports the odds ratio for initiating family planning by 14 weeks postpartum, voucher versus control, as 2.51 (CI 1.31–4.79, P < 0.001), whereas Table 2 reports 5.02 [2.27–11.10] for the same comparison. Table 2's own counts, 149 of 158 (voucher) and 122 of 159 (control), give the crude odds ratio the Methods specify, (149 Γ— 37) / (9 Γ— 122) = 5.02; no 2 Γ— 2 table with these group sizes reproduces 2.51 (1.31–4.79), and that interval excludes 5.02. The abstract therefore understates a secondary-outcome effect by a factor of two, although direction, significance, the primary outcome and the conclusions are unchanged. Separately, the abstract, Results and Discussion print the primary outcome's upper confidence limit as 13.97, where Table 2 prints 18.97 and the counts support about 18–19.

What the paper reports

Methods, "Sample size and statistical analysis":

We compared dichotomous outcomes between study groups by estimating crude odds ratios (ORs) with 95% confidence intervals, and testing for differences between the two groups. We estimated P-values with chi-squared testing using a level of significance of 0.05.

Abstract, "Methods and findings":

By 8 weeks postpartum, family planning was initiated in 144 (91%) participants in the voucher group and 83 (52%) participants in the control group (odds ratio [OR] 9.42; CI 4.67–13.97, P < 0.001). We also found high family planning uptake rates for both groups, with higher rates among the intervention group at 12 weeks (OR 5.66; CI 2.65–12.12, P < 0.001), 14 weeks (OR 2.51; CI 1.31–4.79, P < 0.001), 16 weeks (OR 4.02; CI 1.66–9.77, P = 0.001), and 20 weeks (OR 3.65; CI 1.40–9.47, P = 0.004) postpartum.

Table 2, "Primary and secondary outcomes by treatment group" (columns: Outcomes | Routine Care (n = 159) | FP Voucher (n = 158) | OR (95% CI) | P-Value), rows under "Initiation of FP":

8 weeks postpartum | 83 (52.2%) | 144 (91.4%) | 9.42 [4.67–18.97] | <0.001
12 weeks | 115 (72.3%) | 148 (93.7%) | 5.66 [2.65–12.12] | <0.001
14 weeks | 122 (76.7%) | 149 (94.3%) | 5.02 [2.27–11.10] | <0.001
16 weeks | 134 (84.3%) | 151 (95.6%) | 4.02 [1.66–9.77] | 0.001
20 weeks | 139 (87.4%) | 152 (96.2%) | 3.65 [1.40–9.47] | 0.004

Results, second paragraph:

By 8 weeks postpartum (primary outcome), family planning was initiated in 144 (91%) participants in the family planning voucher group and 83 (52%) participants in the standard of care group (OR 9.42; CI 4.67–13.97; Table 2). Contraceptive use rates continued to increase between 8 and 20 weeks and remained consistently significantly higher among the intervention group.

Discussion, first paragraph:

We found that over 90% of women supported through this family planning voucher accessed family planning by 8 weeks postpartum, which corresponds to 9.42 (CI 4.67–13.97) increased odds of initiating family planning by 8 weeks postpartum.

The 14-week odds ratio is not given as a number anywhere else in the text. The Author summary and Discussion refer to the 12- to 20-week results only qualitatively.

What the paper's own numbers show

Method. For each time point I took the Table 2 counts and built the 2 Γ— 2 table: voucher initiators out of 158 against control initiators out of 159. I then computed the crude (sample) odds ratio, the Woolf (log-Wald) 95% interval, the exact conditional interval, and the Pearson chi-squared P value without continuity correction, which is the test the Methods name. For each interval the paper prints, I also back-calculated the standard error of log OR it implies, its geometric midpoint and the Wald P value. Finally I searched every 2 Γ— 2 table with group sizes 158 and 159 for one that reproduces the abstract's 14-week figures. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
14-week OR Abstract 2.51 (CI 1.31–4.79), P < 0.001 5.02 = (149 Γ— 37) / (9 Γ— 122) = 5513 / 1098
14-week OR Table 2 5.02 [2.27–11.10], P < 0.001 5.02; Woolf 2.33–10.81; exact 2.26–12.25; χ² = 19.73, P = 8.9 Γ— 10-6
Ratio of the two 14-week ORs Table 2 vs Abstract 5.02 / 2.51 2.000
Is 5.02 inside the Abstract's CI? Abstract 1.31–4.79 No
Wald P implied by the Abstract's own OR and CI Abstract P < 0.001 SE(log OR) 0.331, z = 2.78, P = 0.0054
2 Γ— 2 tables (n = 158, 159) giving 2.51 with Woolf CI 1.31–4.79 β€” β€” none
Nearest tables with OR β‰ˆ 2.5 β€” β€” need voucher 141 (not 149) or control 138 (not 122); these contradict the printed 94.3% and 76.7%
Other effect measures from the 14-week counts β€” β€” risk ratio 1.229; inverse OR 0.199; risk difference 17.6 points; none is 2.51
8-week OR (primary) Abstract, Results, Discussion 9.42 (CI 4.67–13.97) 9.42; Woolf 5.01–17.70; exact 4.87–19.07
8-week OR (primary) Table 2 9.42 [4.67–18.97] as above; 18.97 is the log-symmetric mirror of 4.67 around 9.42 (9.42Β² / 4.67 = 19.00)
Geometric midpoint of the 8-week interval Abstract vs Table 2 4.67–13.97 vs 4.67–18.97 8.08 vs 9.41 (point estimate 9.42)
12-, 16-, 20-week ORs Abstract and Table 2 (identical) 5.66 [2.65–12.12]; 4.02 [1.66–9.77]; 3.65 [1.40–9.47] 5.66 (2.73–11.73), P = 4.3 Γ— 10-7; 4.02 (1.69–9.60), P = 0.0008; 3.65 (1.42–9.34), P = 0.0044
ORs from 8 to 20 weeks Abstract 9.42, 5.66, 2.51, 4.02, 3.65 9.42, 5.66, 5.02, 4.02, 3.65 (strictly decreasing)
8-week voucher percentage Table 2 144 (91.4%) 144 / 158 = 91.1% (Abstract and Results say 91%)

At all five time points the Table 2 odds ratios reproduce exactly from the Table 2 counts. So do the P values, to their printed precision. The Table 2 intervals are centred on their odds ratios on the log scale (geometric midpoints 9.41, 5.67, 5.02, 4.03, 3.64). They are slightly wider than the Woolf intervals, with implied standard errors 1.02 to 1.11 times the Woolf values. I could not identify the exact interval procedure used. These small differences are not part of the finding. The abstract's 14-week line is the only one of the five whose point estimate disagrees with the counts. The 8-week upper limit printed in the text is the only one of these initiation intervals that is not symmetric on the log scale around its estimate: it sits 0.70 log units below 9.42 but only 0.39 above.

Figure

The abstract's 14-week odds ratio and interval lie well below the value that Table 2 prints and that its counts reproduce

Figure 1. The odds ratio for initiating family planning by 14 weeks, voucher versus control, on a log scale. Orange rows are the values printed in the paper. Filled: the abstract, 2.51 (1.31–4.79). Open: Table 2, 5.02 (2.27–11.10). Blue rows are recomputed from the Table 2 counts, 149/158 versus 122/159. Filled: the crude odds ratio with Woolf interval, 5.02 (2.33–10.81). Open: the conditional maximum-likelihood estimate with exact interval, 5.00 (2.26–12.25). The dashed blue line marks the recomputed 5.02, which falls outside the abstract's interval. The solid grey line marks no effect (OR = 1). The log scale is the natural one for ratios and does not exaggerate the gap: the abstract value is half the recomputed one.

Across all five time points, only the abstract's 14-week estimate and the abstract's 8-week upper limit depart from the values the counts give

Figure 2. Odds ratios with 95% intervals at each time point, log scale. Orange: as printed in the abstract. Blue: recomputed from the Table 2 counts (Woolf interval). At 12, 16 and 20 weeks the two agree, apart from small differences in interval method. At 14 weeks the abstract's estimate is half the recomputed one, and it breaks an otherwise steady decline from 9.42 to 3.65 as uptake rises in the control arm. At 8 weeks the point estimates agree, but the abstract's upper limit, 13.97, falls short of the recomputed 17.70 and of the 18.97 in Table 2. Labels in bold mark the abstract values that differ from Table 2.

Why it matters

Explanations considered

Status

Suggested correction

In the abstract, replace

14 weeks (OR 2.51; CI 1.31–4.79, P < 0.001)

with

14 weeks (OR 5.02; CI 2.27–11.10, P < 0.001)

so that it matches Table 2 and the counts 149/158 versus 122/159.

In the abstract, the second paragraph of the Results and the first paragraph of the Discussion, replace the 8-week interval "CI 4.67–13.97" with "CI 4.67–18.97", as in Table 2. The authors may also wish to change the 8-week voucher percentage in Table 2 from 91.4% to 91.1% (144/158), and to state which interval method Table 2 uses, since its limits differ slightly from the Woolf intervals for the same counts.