Paperonium

← Back to the plain-language case

Abstract reports group 2's 1-year vaccine efficacy, 77% (67–84), as the value for group 1

Lancet (2021) Β· doi:10.1016/S0140-6736(21)00943-0 Β· PMC8121760

Summary

The abstract of this phase 2b trial of the R21/Matrix-M malaria vaccine has one 1-year efficacy figure: "77% (67–84) in group 1", where group 1 is the lower adjuvant dose (25 ΞΌg Matrix-M). Results paragraph 3 (unadjusted) and Table 1 (unadjusted and adjusted) give group 1 a 12-month efficacy of 71% (59–79). The value 77% (67–84) is group 2's 12-month estimate (50 ΞΌg Matrix-M), matching to the point estimate and both confidence limits, and no group-1 estimate anywhere in the paper equals it. The paper reads efficacy against a WHO goal of at least 75%. Read alone, the abstract places the lower-dose arm above that goal at 1 year and has its efficacy rising from 74% at 6 months, when the paper's own data show it below the goal and falling.

What the paper reports

Abstract, Findings:

Vaccine efficacy was 74% (95% CI 63–82) in group 1 and 77% (67–84) in group 2 at 6 months. At 1 year, vaccine efficacy remained high, at 77% (67–84) in group 1.

Abstract, Methods (the group definitions):

Group 1 received 5 ΞΌg R21 plus 25 ΞΌg MM, group 2 received 5 ΞΌg R21 plus 50 ΞΌg MM, and group 3, the control group, received rabies vaccinations.

Results, second paragraph (6 months, primary endpoint):

These cases of clinical malaria occurred in 43 (29%) of 146 participants in group 1, 38 (26%) of 146 participants in group 2, and 105 (71%) of 147 participants in group 3. A Cox regression model comparing group 1 with group 3 resulted in vaccine efficacy of 74% (95% CI 63–82; p<0Β·0001). Comparing group 2 with group 3 resulted in 77% efficacy (67–84; p<0Β·0001; figure 2).

Results, third paragraph (12 months):

These cases occurred in 50 (34%) of 146 participants in group 1, 39 (27%) of 146 participants in group 2, and 106 (72%) of 147 participants in group 3. Cox regression showed vaccine efficacy of 71% (95% CI 59–79; p<0Β·0001) for group 1, and 77% (67–84; p<0Β·0001) for group 2 (figure 2).

Table 1, primary case definition (>5000 parasites per ΞΌL), columns: timepoint | cases | unadjusted efficacy | adjusted efficacy:

Group 1 … 12 | 50/146 (34%) | 71% (59–79) | 71% (59–79)

Group 2 … 12 | 39/146 (27%) | 77% (67–84) | 76% (65–84)

Discussion, first paragraph (reference number after "75% or more" omitted):

We report malaria vaccine efficacy that reaches the WHO-specified efficacy goal of 75% or more in the target population of African children, over 12 months of follow-up. Efficacy of 77% was observed in the children who received R21 plus the higher dose of MM adjuvant

Discussion, seventh paragraph:

Efficacy with R21 and the higher adjuvant MM dose, administered largely before the malaria season, was 77% (95% CI 67–84), compared with 44% (37–50) reported for RTS,S/AS01 at this site

Research in context, Added value of this study:

This new vaccine, R21 adjuvanted with 50 ΞΌg Matrix-M (R21/MM), administered before the malaria season, demonstrates high-level efficacy, reaching the WHO-specified efficacy goal of at least 75% in the target population of African children over 1 year.

What the paper's own numbers show

Method. Table 1 states that efficacy is 1 minus the Cox hazard ratio. I made three checks, all printed by recompute.py.txt. (1) I compared the abstract's triplet, 77% (67–84), with every group-1 estimate printed in the paper and with the group-2 estimates. (2) Cox estimates cannot be reproduced exactly without event times, so as an independent check I approximated each efficacy from the printed case counts as 1 minus the ratio of cumulative hazards, with H = βˆ’ln(1 βˆ’ cases/N) and SE(log HR) β‰ˆ √(1/d₁ + 1/dβ‚‚). This approximation runs 1.1 to 3.8 points below Table 1 in all four cells, so it is useful for direction and size, not for the exact value. (3) I counted the new first cases between 6 and 12 months among the children still case-free at 6 months.

Quantity Location Reported Recomputed
Group 1, 12 months Abstract 77% (67–84) does not match any group-1 estimate in the paper
Group 1, 12 months, unadjusted Results para 3; Table 1 71% (59–79) HR 0.29 (0.21–0.41); count-based approximation 67.2% (54–77)
Group 1, 12 months, adjusted Table 1 71% (59–79) not 77% (67–84)
Group 2, 12 months, unadjusted Results para 3; Table 1; Discussion 77% (67–84) identical to the abstract's "group 1" triplet; HR 0.23 (0.16–0.33); approximation 75.7% (65–83)
Group 1, 6 months Abstract; Results para 2; Table 1 74% (63–82) approximation 72.2% (60–80)
Group 2, 6 months Abstract; Results para 2; Table 1 77% (67–84) approximation 75.9% (65–83)
New first cases, 6 to 12 months Results; Table 1 (43β†’50, 38β†’39, 105β†’106) β€” group 1: 7 of 103 still at risk (6.8%); group 2: 1 of 108 (0.9%); control: 1 of 42 (2.4%)
Change in group 1 efficacy, 6 to 12 months Abstract vs Table 1 abstract implies 74% β†’ 77% Table 1 74% β†’ 71%; approximation 72.2% β†’ 67.2% (βˆ’5.0 points)
Other group-1 estimates (secondary definition; Results para 5 adjusted; groups 1+2 pooled) Table 1; Results paras 4–5 70% to 75%, CIs 60–81 none equals 77% (67–84)
Group 1 and group 2, 12 months, all episodes (negative binomial rate ratio; a different measure from the Cox first-episode estimate) Appendix p 2, Supplementary Table 2b 69.6% (58.7–77.6) and 75.7% (66.3–82.4) group 1 below 75% and below group 2; neither equals 77% (67–84)
Overstatement of group 1 at 12 months Abstract β€” 6 percentage points; crosses the 75% goal

Group 1 gained seven new cases between 6 and 12 months against one in the control group. Over that interval the risk in group 1 was higher than in the controls who were still case-free, so the 12-month estimate for group 1 should be lower than its 6-month estimate. Table 1 shows that fall (74% to 71%). The abstract's statement that group 1 "remained high, at 77%" would need a rise.

Figure

The abstract's 1-year value for group 1, 77% (67–84), is the Results and Table 1 value for group 2; group 1's own value is 71% (59–79)

Figure 1. Left: the abstract's only 1-year efficacy sentence. Right: the 12-month values for the two vaccine arms as printed in Results paragraph 3 and Table 1 (primary case definition, unadjusted Cox). The solid arrow links the abstract's triplet to the identical group-2 row; the dashed arrow links it to the group-1 row, which has the same label and a different value. Orange text is printed in the paper. Blue text at the bottom is recomputed by recompute.py.txt from the paper's case counts as 1 minus the ratio of cumulative hazards (an approximation to the Cox estimate).

Vaccine efficacy by group and time point: Table 1 estimates, the abstract's value, and count-based approximations, on one axis from 0 to 100%

Figure 2. Vaccine efficacy (%, 95% CI) on one linear axis from 0 (no effect, solid line) to 100, with the WHO goal of 75% as a dashed line. Orange circles: Table 1 unadjusted Cox estimates. Open orange diamond with dashed interval: the abstract's 1-year value for group 1. Blue squares: approximations recomputed from the printed case counts. The abstract's value sits with the group-2 rows, not with group 1, whose estimate falls between 6 and 12 months by both the Cox and the count-based measure.

Why it matters

Explanations considered

Status

Suggested correction

In the abstract, Findings, replace

At 1 year, vaccine efficacy remained high, at 77% (67–84) in group 1.

with

At 1 year, vaccine efficacy was 71% (59–79) in group 1 and 77% (67–84) in group 2.

This agrees with Results paragraph 3 and Table 1. The authors may also wish to reconcile the adjusted 12-month estimates in Results paragraph 5, 75% (67–81) and 77% (65–80), with Table 1's adjusted column, 71% (59–79) and 76% (65–84), and the 29% versus 30% rounding of 43/146.