Paperonium

โ† Back to the plain-language case

Abstract pairs the adjusted BNT162b2 estimate with the interval of the unadjusted ChAdOx1 estimate

Lancet Infect Dis (2021) ยท doi:10.1016/S1473-3099(21)00330-3 ยท PMC8221734

Summary

The abstract reports the adjusted one-dose effectiveness of BNT162b2 against COVID-19 hospitalisation in adults aged at least 80 years as 71ยท4% (95% CI 46ยท5โ€“90ยท6). The Results text and Table 2 give the same estimate as 71ยท4% (43ยท1โ€“86ยท2), and that interval follows exactly from the adjusted odds ratio printed in Table 2, 0ยท286 (0ยท138โ€“0ยท569), since the paper defines effectiveness as 1 โˆ’ OR. The interval in the abstract, 46ยท5โ€“90ยท6, is identical, both as effectiveness and as odds ratio (0ยท094โ€“0ยท535), to the interval of a different estimate: the unadjusted ChAdOx1 nCoV-19 effectiveness of 76ยท7%. The point estimate and the conclusion are unaffected, but a headline sentence of the abstract gives the estimate an interval that is shifted upwards by 3.4 points at the lower end and 4.4 at the upper end, wider on the log-odds scale (implied SE 23% larger), and not centred on its own point estimate. The error is in the version of record and has not been corrected.

What the paper reports

Abstract, Findings (BNT162b2):

18 (13%) of 135 people with SARS-CoV-2 infection and 90 (34%) of 269 controls received one dose of BNT162b2. The adjusted vaccine effectiveness was 71ยท4% (95% CI 46ยท5โ€“90ยท6).

Abstract, Findings (ChAdOx1 nCoV-19 and the period-restricted BNT162b2 estimate):

Nine (25%) of 36 people with COVID-19 infection and 53 (59%) of 90 controls received one dose of ChAdOx1 nCoV-19. The adjusted vaccine effectiveness was 80ยท4% (95% CI 36ยท4โ€“94ยท5). When BNT162b2 effectiveness analysis was restricted to the period covered by ChAdOx1 nCoV-19, the estimate was 79ยท3% (95% CI 47ยท0โ€“92ยท5).

Results, third paragraph (BNT162b2):

18 (13%) of the 135 people with SARS-CoV-2 infection and 90 (34%) of 269 controls received one dose BTN162b2 (difference -20ยท2%), giving an unadjusted vaccine effectiveness of 69ยท4% (95% CI 47ยท7โ€“82ยท9) and an adjusted effectiveness of 71ยท4% (43ยท1โ€“86ยท2; table 2 ).

Results, fifth paragraph (ChAdOx1 nCoV-19):

Nine (25%) of the 36 people with SARS-CoV-2 infection and 53 (59%) of 90 controls received one dose of ChAdOx1 nCoV-19 (difference -33ยท9%), giving an unadjusted vaccine effectiveness of 76ยท7% (95% CI 46ยท5โ€“90ยท6) and adjusted effectiveness of 80ยท4% (36ยท4โ€“94ยท5; table 2).

Table 2, "One dose of BNT162b2 (Pfizer)", logistic regression model, row "One dose" (vaccine effectiveness, odds ratio, p value):

71ยท4 (43ยท1 to 86ยท2) | 0ยท286 (0ยท138โ€“0ยท569) | <0ยท0001

Table 2, "One dose of ChAdOx1 nCoV-19 (Oxfordโ€“AstraZeneca)", row "Unadjusted":

76ยท7 (46ยท5 to 90ยท6) | 0ยท233 (0ยท094โ€“0ยท535) | <0ยท0001

What the paper's own numbers show

Method. The paper defines vaccine effectiveness as (1 โˆ’ OR) ร— 100, and all eight one-dose effect rows of Table 2 obey that relation to the printed precision, except for a 0.1-point rounding difference in the upper bound of the ChAdOx1 matched-conditional row (printed 93ยท2; 1 โˆ’ 0ยท067 gives 93.3). So any effectiveness interval can be converted back to an odds-ratio interval and compared with the table. For the adjusted estimate the odds ratio cannot be recomputed from raw counts, but its effectiveness interval follows directly from the printed OR interval. For the unadjusted estimates, the 2 ร— 2 counts given in the abstract and Results allow an independent recomputation (Wald interval on the log-odds scale, and the conditional maximum-likelihood estimate with its exact interval). A further check uses the fact that a 95% Wald interval is symmetric about ln(OR): I measure how far each point estimate lies from the log-midpoint of its interval, in units of the interval's half-width. All numbers below are printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
BNT162b2 adjusted VE (95% CI) Abstract 71ยท4% (46ยท5โ€“90ยท6) 71.4 (43.1โ€“86.2) from Table 2 OR 0ยท286 (0ยท138โ€“0ยท569)
BNT162b2 adjusted VE (95% CI) Results ยถ3; Table 2 71ยท4 (43ยท1 to 86ยท2) 71.4 (43.1โ€“86.2), agrees
Abstract BNT162b2 interval, as OR Abstract 46ยท5โ€“90ยท6 OR 0.094โ€“0.535
ChAdOx1 unadjusted VE and OR Results ยถ5; Table 2 76ยท7 (46ยท5 to 90ยท6); OR 0ยท233 (0ยท094โ€“0ยท535) 76.7 (46.5โ€“90.6), agrees; same interval as the abstract's BNT162b2 interval
Table 2 rows whose VE interval is 46ยท5 to 90ยท6 Table 2 โ€” only "ChAdOx1 unadjusted"
Log-midpoint of the abstract BNT162b2 interval Abstract โ€” OR 0.224 (VE 77.6), vs point estimate OR 0.286
Point estimate's distance from its interval's log-midpoint Abstract, BNT162b2 adjusted โ€” +0.28 half-widths
Same distance, all eight Table 2 effect rows and the other two abstract intervals Table 2; Abstract โ€” between 0.00 and +0.04 half-widths
BNT162b2 unadjusted OR from counts 18/135 vs 90/269 Counts: Abstract, Results ยถ3; reported OR/VE: Table 2 (VE also Results ยถ3) 0ยท306 (0ยท171โ€“0ยท523); VE 69ยท4 (47ยท7โ€“82ยท9) Wald 0.306 (0.175โ€“0.534), VE 69.4 (46.6โ€“82.5); conditional MLE 0.307 (0.165โ€“0.545), VE 69.3 (45.5โ€“83.5)
ChAdOx1 unadjusted OR from counts 9/36 vs 53/90 Counts: Abstract, Results ยถ5; reported OR/VE: Table 2 (VE also Results ยถ5) 0ยท233 (0ยท094โ€“0ยท535); VE 76ยท7 (46ยท5โ€“90ยท6) Wald 0.233 (0.098โ€“0.552), VE 76.7 (44.8โ€“90.2); conditional MLE 0.235 (0.087โ€“0.589), VE 76.5 (41.1โ€“91.3)
ChAdOx1 adjusted VE Abstract; Table 2 80ยท4% (36ยท4โ€“94ยท5); OR 0ยท196 (0ยท055โ€“0ยท636) 80.4 (36.4โ€“94.5), agrees
BNT162b2 restricted-period VE Abstract 79ยท3% (47ยท0โ€“92ยท5) OR 0.207 (0.075โ€“0.530); centred (+0.04 half-widths)
SE(ln OR) implied by the interval Abstract vs Table 2 โ€” 0.444 (abstract) vs 0.361 (Table 2), 23% larger

The Table 2 conversion holds to the printed precision for every row except the 0.1-point rounding difference noted above (ChAdOx1 matched conditional, 93ยท2 vs 93.3), and exactly for the BNT162b2 adjusted row, so the correct interval for 71ยท4% is 43ยท1โ€“86ยท2. The abstract's interval matches the ChAdOx1 unadjusted interval digit for digit, and its log-midpoint (OR 0.224) is that row's midpoint, not the midpoint of the BNT162b2 adjusted interval (OR 0.280).

Figure

The abstract's BNT162b2 interval has exactly the endpoints of the unadjusted ChAdOx1 interval and is not centred on its own point estimate

Figure 1. Odds ratios for one dose versus none, on a log scale (VE = 1 โˆ’ OR), with 95% intervals. Orange circles and lines are values printed in the paper (the Abstract row shows the abstract's printed VE converted to OR, 1 โˆ’ VE); blue squares and lines are unadjusted odds ratios recomputed by recompute.py.txt from the paper's 2 ร— 2 counts (Wald intervals); the open black diamond is the log-midpoint of the abstract's interval, computed by recompute.py.txt. The right-hand column gives each row as VE % (95% CI), in the colour of its row. The abstract row (top) carries the BNT162b2 adjusted point estimate, 0.286, but its interval ends exactly where the ChAdOx1 unadjusted interval ends (dashed lines at 0.094 and 0.535), and its centre (0.224) lies well to the left of the point estimate. The Table 2 BNT162b2 rows and the interval recomputed from the BNT162b2 counts end at 0.138โ€“0.175 at the lower OR bound (adjusted 0.138, unadjusted 0.171, Wald from counts 0.175), not at 0.094. The vertical line marks no effect (OR 1).

Two text panels quoting the abstract and Table 2 for BNT162b2, joined by a connector, and a third panel showing the ChAdOx1 unadjusted interval that matches the abstract

Figure 2. The three places in the paper that give the conflicting numbers. The abstract (orange outline) and Results/Table 2 (grey outline, top right) give the same BNT162b2 point estimate with different intervals; the abstract's interval is identical to the unadjusted ChAdOx1 nCoV-19 interval in Results paragraph 5 and Table 2 (bottom right). Orange text is printed in the paper; blue text is the 1 โˆ’ OR conversion computed by recompute.py.txt.

Why it matters

Explanations considered

Status

Suggested correction

In the abstract (Findings), replace

The adjusted vaccine effectiveness was 71ยท4% (95% CI 46ยท5โ€“90ยท6).

with

The adjusted vaccine effectiveness was 71ยท4% (95% CI 43ยท1โ€“86ยท2).

so that the abstract agrees with the Results text, Table 2 and the adjusted odds ratio 0ยท286 (0ยท138โ€“0ยท569). The same change would apply to the PubMed abstract and could be noted against the accompanying Comment, which quotes the abstract's interval. No other number in the abstract needs changing.