Paperonium

← Back to the plain-language case

Prediabetes meta-analysis reports mortality risk difference 7.36 outside its own 95% CI (9.59 to 12.51)

BMJ (2020) Β· doi:10.1136/bmj.m2297 Β· PMC7362233

Summary

The abstract and the Results both give the absolute risk difference (ARD) for all cause mortality with prediabetes in the general population as "7.36 (95% confidence interval 9.59 to 12.51)" per 10 000 person years, so the estimate lies outside its own interval. The paper derives each ARD as comparator risk Γ— (RR βˆ’ 1). Its supplement gives the comparator risk as 73.6 per 10 000 person years, and the pooled relative risk is 1.13 (1.10 to 1.17), so the ARD works out to 9.57 (7.36 to 12.51). The printed estimate and the printed lower limit have swapped places. The other seven ARDs in the paper reproduce exactly by the same method. The absolute excess mortality the abstract gives is therefore about 23% too low (7.36 rather than about 9.6 per 10 000 person years), while the pooled relative risk and the paper's conclusions are unaffected.

What the paper reports

Abstract, Results, and again word for word in Results, section "All cause mortality and cardiovascular disease associated with prediabetes in the general population", third paragraph:

Compared with normoglycaemia, the absolute risk difference in prediabetes for all cause mortality, composite cardiovascular disease, coronary heart disease, and stroke was 7.36 (95% confidence interval 9.59 to 12.51), 8.75 (6.41 to 10.49), 6.59 (4.53 to 8.65), and 3.68 (2.10 to 5.26) per 10 000 person years, respectively.

Abstract, Results, with the relative risks these ARDs are derived from:

In the general population, prediabetes was associated with an increased risk of all cause mortality (relative risk 1.13, 95% confidence interval 1.10 to 1.17), composite cardiovascular disease (1.15, 1.11 to 1.18), coronary heart disease (1.16, 1.11 to 1.21), and stroke (1.14, 1.08 to 1.20) in a median follow-up time of 9.8 years.

Methods, "Statistical analysis", which defines the calculation:

We calculated the absolute risk difference for all cause mortality and cardiovascular outcomes associated with prediabetes by multiplying the assumed comparator risk of each outcome of interest by the estimated RR-1, according to the recommendation of the Cochrane guidelines.25 We used the median risks of outcomes in individuals with normoglycaemia across studies as the assumed comparator risks. Absolute risk differences were expressed as events per 10 000 person years.

Supplementary file 15 (supplement PDF, p. 54), "Men and women" row for all cause mortality. The columns are "Event rate in normoglycaemia (No of studies)" and "Absolute risk difference":

All cause mortality 73.6 (44) 7.36 (95% CI 9.59 to 12.51)

What the paper's own numbers show

Method. Under the stated method, the estimate and both 95% limits of an ARD are the comparator risk C multiplied by (RR βˆ’ 1), (RR lower βˆ’ 1) and (RR upper βˆ’ 1), always with the same C. Two checks follow. (1) Multiply the supplement's C by the printed relative risks. (2) Divide each printed ARD number by its (RR βˆ’ 1) and see whether it gives back the supplement's C. Because the RRs are printed to two decimals, each implied C carries a tolerance, worked out by moving the RR by Β±0.005. I ran both checks on all eight ARD triples in the paper: four for the general population (supplementary file 15) and four for patients with atherosclerotic cardiovascular disease (ASCVD; supplementary file 21). Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
Comparator risk C, all cause mortality, general population Supplementary file 15 73.6 per 10 000 PY (44 studies) input
RR, all cause mortality, general population Abstract; Results 1.13 (1.10 to 1.17) input
ARD, all cause mortality, general population Abstract; Results; Supp. file 15 7.36 (9.59 to 12.51) 9.57 (7.36 to 12.51) = 73.6 Γ— (0.13; 0.10; 0.17)
Is the estimate inside its own CI? same 7.36 < 9.59 no; 2.23 below the printed lower limit
C implied by printed estimate 7.36 / 0.13 same β€” 56.6 (rounding range 54.5 to 58.9); does not fit 73.6
C implied by printed lower limit 9.59 / 0.10 same β€” 95.9 (rounding range 91.3 to 100.9); does not fit 73.6
C implied by printed upper limit 12.51 / 0.17 same β€” 73.6; fits
C implied by transposed triple 9.59 (7.36 to 12.51) β€” β€” 73.8 / 73.6 / 73.6; all fit
RR implied by 9.59 with C = 73.6 β€” β€” 1.1303, i.e. an unrounded RR that prints as 1.13
ARD, composite CVD, general population (C = 58.3) Abstract; Results 8.75 (6.41 to 10.49) 8.74 (6.41 to 10.49)
ARD, coronary heart disease, general population (C = 41.2) Abstract; Results 6.59 (4.53 to 8.65) 6.59 (4.53 to 8.65)
ARD, stroke, general population (C = 26.3) Abstract; Results 3.68 (2.10 to 5.26) 3.68 (2.10 to 5.26)
ARD, all cause mortality, ASCVD (C = 183.8) Abstract; Results 66.19 (38.60 to 99.25) 66.17 (38.60 to 99.25)
ARD, composite CVD, ASCVD (C = 512.9) Abstract; Results 189.77 (117.97 to 271.84) 189.77 (117.97 to 271.84)
ARD, coronary heart disease, ASCVD (C = 270.8) Abstract; Results 40.62 (5.42 to 78.53) 40.62 (5.42 to 78.53)
ARD, stroke, ASCVD (C = 170.7) Results; Supp. file 21 8.54 (βˆ’32.43 to 61.45) 8.54 (βˆ’32.43 to 61.45)
Same, as printed in the abstract Abstract 8.54 (32.43 to 61.45) lower limit is βˆ’32.43; the minus sign is missing
Printed numbers consistent with the supplement's C all 8 triples 24 numbers 22 of 24; the two exceptions are the general-population mortality estimate and lower limit

The method reproduces 22 of the 24 printed ARD numbers to within 0.02, which confirms that the paper's ARDs were computed exactly as its Methods describe. The only two that fail are the estimate and lower limit of a single triple, and exchanging those two makes that triple consistent too.

Figure

Printed general-population absolute risk differences compared with values recomputed as the supplement's comparator rate times (RR βˆ’ 1); only all cause mortality differs, with the printed estimate outside its interval

Figure 1. Absolute risk difference of prediabetes against normoglycaemia in the general population, per 10 000 person years, for the four outcomes in the abstract. Orange: the estimate and 95% CI as printed in the abstract and Results. Blue: recomputed by recompute.py.txt as the comparator event rate from supplementary file 15 Γ— (RR βˆ’ 1), using the printed relative risks and their limits. The numbers are given at the right. For composite CVD, coronary heart disease and stroke the printed and recomputed intervals coincide. For all cause mortality the printed estimate (7.36) lies to the left of its own interval, and the recomputed estimate, 9.57, lies inside the same interval, which runs from 7.36 to 12.51. The vertical line at 0 marks no difference; the axis starts at that line.

For each of the 24 printed ARD numbers, the comparator risk it implies relative to the supplement's stated rate; 22 sit at 1, the mortality estimate sits at 0.77 and its lower limit at 1.30

Figure 2. Each printed ARD number divided by its (RR βˆ’ 1) gives the comparator risk it implies. The figure plots that implied risk as a ratio to the event rate stated in supplementary files 15 and 21, for all eight outcomes (circle: estimate; left triangle: lower limit; right triangle: upper limit). Grey markers fit the stated rate within the tolerance allowed by two-decimal rounding of the RR; all 22 of them lie at 1.00 (range 0.998 to 1.001). The two orange markers are printed values that do not fit: the general-population mortality estimate 7.36 implies 0.77 times the stated rate, and its lower limit 9.59 implies 1.30 times. Exchanging those two values puts both at 1.00.

Why it matters

Explanations considered

Status

As of 2026-10-24.

Suggested correction

In the abstract (Results) and in the Results section, "All cause mortality and cardiovascular disease associated with prediabetes in the general population", paragraph 3, and in supplementary file 15 (men and women, all cause mortality):

In the abstract only: