Paperonium

← Back to the plain-language case

Risk-difference confidence intervals for the primary outcome are about 23 to 31 times too narrow

JAMA Netw Open (2026) · doi:10.1001/jamanetworkopen.2026.25965 · PMC13434869

Summary

The Abstract and the Results report the trial's primary-outcome effect sizes, the absolute differences in first-pass intubation success between direct laryngoscopy (DL), Macintosh-blade video laryngoscopy (VLM) and hyperangulated video laryngoscopy (VLH), with 95% confidence intervals 0.22 to 0.33 percentage points wide, for example 4.66 (95% CI, 4.45-4.78) for VLM vs DL (written "DL vs VLM" in the paper). The paper's own Table 2 counts (625/799, 673/812, 711/812) reproduce every point estimate and, to within 0.03, the paper's own z statistics (exactly with a pooled standard error), but with the method the paper names (an unpooled z approach), they give intervals 6.9 to 7.7 points wide: 0.80 to 8.52, 5.69 to 12.99 and 1.24 to 8.12. The numbers needed to treat in the Discussion carry the same narrow intervals (VLM 21.5, reported 20.9-22.2; from the counts, 11.7-125.2). The direction and statistical significance of every comparison are unchanged, but the published intervals overstate the precision of the trial's headline result 23- to 31-fold. That matters most for VLM, where the data are compatible with a gain of anywhere from 0.8 to 8.5 points.

What the paper reports

Differences between success rates were tested applying the z test for unpooled variance and reported as absolute differences with corresponding 95% CIs.

Abstract, Main Outcomes and Measure

Unadjusted absolute risk differences for first-pass intubation success were 4.66 (95% CI, 4.45-4.78) percentage points for DL vs VLM, 9.34 (95% CI, 9.22-9.46) percentage points for DL vs VLH, and 4.68 (95% CI, 4.57-4.79) percentage points for VLM vs VLH.

Abstract, Results

In 2-sided z tests, VLM and VLH were significantly more effective in facilitating first-pass success than the DL (VLM vs DL: z = 2.36, P = .02; VLH vs DL: z = 4.98, P < .001). VLH was in turn superior to VLM (z = 2.66; P = .008). Unadjusted absolute risk differences for a successful first attempt were 4.66 (95% CI, 4.45-4.78) percentage points for DL vs VLM, 9.34 (95% CI, 9.22-9.46) percentage points for DL vs VLH, and 4.68 (95% CI, 4.57-4.79) percentage points for VLM vs VLH.

Results, Primary End Point and Crossover Between Groups

Relative risks (RRs) for successful first-attempt intubation were significantly higher for both VLM and VLH over DL (VLM vs DL: RR, 1.06 [95% CI, 1.01-1.11]; VLH vs DL: RR, 1.12 [95% CI, 1.07-1.17]) and for VLM over VLH (RR, 1.06 [95% CI, 1.01-1.10]) (eTable 2 in Supplement 1).

Results, same paragraph

Numbers needed to treat were 21.5 (95% CI, 20.9-22.2) for VLM and 10.7 (95% CI, 10.6-10.8) for VLH.

Discussion

Table 2 ("First-Pass Success Rates of Tracheal Intubations Using Different Laryngoscopes"), mITT population, row "Yes": DL 625 (78.2), VLM 673 (82.9), VLH 711 (87.6); row "Total": 799, 812, 812.

What the paper's own numbers show

Method. For each pair of arms I took the Table 2 counts and computed the risk difference, its unpooled standard error √(p₁(1−p₁)/n₁ + p₂(1−p₂)/n₂), and the 95% Wald interval RD ± 1.96·SE, which is the interval that goes with the unpooled z test the paper says it used. As checks I also computed the Newcombe (hybrid Wilson score) and Agresti–Caffo intervals, the z statistics (unpooled and pooled), the log-scale Wald interval for the relative risk, and the NNT interval obtained by inverting the risk-difference limits. Finally, I worked backwards from the reported intervals: dividing a reported width by 2 × 1.96 gives the standard error that interval implies, and dividing the reported risk difference by the reported z gives a second implied standard error. Comparisons are labelled higher-success arm vs lower-success arm (VLM vs DL, VLH vs DL, VLH vs VLM); the paper writes them as DL vs VLM, DL vs VLH and VLM vs VLH, with the same positive values. All numbers below are printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
First-pass success, DL / VLM / VLH Table 2 625/799 (78.2), 673/812 (82.9), 711/812 (87.6) 78.22%, 82.88%, 87.56%
RD, VLM vs DL (pp) Abstract; Results 4.66 4.66
95% CI, VLM vs DL Abstract; Results 4.45 to 4.78 (width 0.33) Wald 0.80 to 8.52 (width 7.72); Newcombe 0.79 to 8.52; Agresti–Caffo 0.79 to 8.51
RD, VLH vs DL (pp) Abstract; Results 9.34 9.34
95% CI, VLH vs DL Abstract; Results 9.22 to 9.46 (width 0.24) Wald 5.69 to 12.99 (width 7.31); Newcombe 5.68 to 12.99; Agresti–Caffo 5.66 to 12.97
RD, VLH vs VLM (pp) Abstract; Results 4.68 4.68
95% CI, VLH vs VLM Abstract; Results 4.57 to 4.79 (width 0.22) Wald 1.24 to 8.12 (width 6.89); Newcombe 1.23 to 8.13; Agresti–Caffo 1.22 to 8.12
Recomputed width ÷ reported width derived — 23.4×, 30.4×, 31.3×
z, VLM vs DL Results 2.36 (P = .02) unpooled 2.37 (P = .018); pooled 2.36
z, VLH vs DL Results 4.98 (P < .001) unpooled 5.01 (P = 5.4 × 10⁻⁷); pooled 4.98
z, VLH vs VLM Results 2.66 (P = .008) unpooled 2.66 (P = .0077); pooled 2.66
SE implied by reported RD ÷ reported z (pp) derived 1.975, 1.876, 1.759 unpooled SE from counts 1.970, 1.864, 1.757
SE implied by reported CI width ÷ 3.92 (pp) derived 0.084, 0.061, 0.056 —
z that the reported CIs would imply derived — 55.4, 152.6, 83.4
Per-arm sample size at which a Wald CI would be as narrow as reported derived — about 441,000, 746,000 and 796,000 (trial: 799–812)
Reported CI, distance below / above point Abstract; Results VLM vs DL −0.21 / +0.12; others −0.12 / +0.12 and −0.11 / +0.11 Wald is symmetric (±3.86, ±3.65, ±3.44)
RR, VLM vs DL Results; eTable 2 1.06 (1.01–1.11) 1.060 (1.010–1.112)
RR, VLH vs DL Results; eTable 2 1.12 (1.07–1.17) 1.119 (1.070–1.171)
RR, VLH vs VLM Results; eTable 2 1.06 (1.01–1.10) 1.056 (1.014–1.100)
NNT, VLM vs DL Discussion 21.5 (20.9–22.2) 21.5 (11.7–125.2)
NNT, VLH vs DL Discussion 10.7 (10.6–10.8) 10.7 (7.7–17.6)
NNT limits from inverting the reported RD CIs derived — VLM 20.9–22.5; VLH 10.6–10.8

Two independent routes give the same standard errors. The counts give 1.97, 1.86 and 1.76 points, and the paper's own risk differences divided by its own z statistics give 1.98, 1.88 and 1.76 points. The reported intervals imply standard errors of 0.06 to 0.08 points, about 23 to 31 times smaller. The relative-risk intervals in the same paragraph and in eTable 2 match the counts to the printed decimal, so the precision the paper reports on the ratio scale agrees with the wide intervals, not the narrow ones. The NNT intervals match the reciprocals of the reported risk-difference limits (exactly for VLH; for VLM the upper limit is 22.2 against 22.5), so they inherit the same error.

Figure

Reported vs recomputed 95% CIs for the three first-pass success risk differences

Figure 1. Each row is one of the three primary-outcome comparisons. Orange diamonds show the risk difference and 95% CI reported in the Abstract and Results. The reported intervals are narrower than the marker, so the bars are hidden behind it; the right-hand labels give their limits. Blue circles with capped bars show the 95% CI recomputed from the Table 2 counts with an unpooled Wald interval, the method the paper states. The solid gray line marks no difference and the dashed line marks the −5 point noninferiority margin. Every recomputed interval lies to the right of both, so significance and noninferiority are unchanged, but each is 23 to 31 times wider than reported. The axis is linear and includes zero.

Reported vs recomputed 95% CIs for the numbers needed to treat

Figure 2. Numbers needed to treat with VLM or VLH instead of DL to gain one additional first-pass success, as reported in the Discussion (orange diamonds) and as obtained by inverting the recomputed Wald risk-difference limits (blue circles). The axis is logarithmic because the NNT is a reciprocal. For VLM, the interval the counts support runs from about 12 to about 125 intubations, against a reported 20.9 to 22.2.

Why it matters

Explanations considered

Two smaller observations, which do not affect the main finding:

Status

As of 2026-10-23:

Sources: the article in PubMed Central (PMC13434869) and at doi:10.1001/jamanetworkopen.2026.25965; Supplements 1, 2 and 4 from the Europe PMC supplementary files; ClinicalTrials.gov NCT05228288; Crossref and Europe PMC metadata.

Suggested correction

Replace the risk-difference intervals in the Abstract and in the Results (Primary End Point and Crossover Between Groups) with intervals computed from the Table 2 counts by the stated method. With the unpooled Wald interval these are:

and replace the Discussion's NNT intervals with 21.5 (95% CI, 11.7–125.2) for VLM and 10.7 (95% CI, 7.7–17.6) for VLH. If a score-based interval is preferred, the Newcombe limits (0.79–8.52, 5.68–12.99, 1.23–8.13) differ only in the second decimal. Optionally, the authors could also report the noninferiority z statistics against the −5 point margin, state whether the reported z statistics used pooled or unpooled variance, and change "for VLM over VLH" to "for VLH over VLM".