Paperonium

← Back to the plain-language case

Hazard ratio for symptomatic COVID-19 (0.011) lies outside its own 95% confidence interval (0.063–0.198)

Kidney Int (2022) · doi:10.1016/j.kint.2022.07.008 · PMC9297679

Summary

This research letter compares 333 kidney transplant recipients (KTRs) who received tixagevimab–cilgavimab preexposure prophylaxis with 97 who did not. For its first outcome, symptomatic COVID-19, it reports a hazard ratio of 0.011 with a 95% CI of 0.063–0.198. A point estimate cannot lie below its own confidence interval. The geometric centre of the interval is 0.112, about ten times the printed value. The paper's other three HRs each sit at the centre of their intervals (ratios 0.997 to 1.027), which points to a dropped digit, with 0.011 printed for about 0.11. Direction and significance do not change. The printed value does overstate the effect size for the key outcome: it implies about a 99% lower hazard, while the interval implies 80–94%, about 89% at its centre.

What the paper reports

Main text, third paragraph (the letter has no abstract and no separate Results heading):

Among the 333 KTRs who received tixagevimab–cilgavimab, 12.3% developed symptomatic COVID-19 compared with 43.3% among those who did not (hazard ratio [HR], 0.011; 95% confidence interval [CI], 0.063–0.198; P < 0.001; Figure 1a).

The same paragraph, for the other three outcomes:

(HR, 0.046; 95% CI, 0.013–0.158; P < 0.001; Figure 1b)

(HR, 0.045; 95% CI, 0.008–0.240; P < 0.001; Figure 1c)

(HR, 0.076; 95% CI, 0.005–1.161; P = 0.066; Figure 1d)

Table 1, row "Symptomatic COVID-19" (columns: treated, n = 333 | untreated, n = 97; n (%)):

Symptomatic COVID-19 | 41 (12.3) | 42 (43.3)

Figure 1 legend:

Log-rank test was used to compare the 2 groups and was considered as significant when P < 0.05.

Main text, fourth paragraph:

The first conclusion of this real-life study is that tixagevimab–cilgavimab use was associated with (i) a significantly lower risk of symptomatic COVID-19 and (ii) fewer COVID-19–related hospitalizations (including intensive care unit), compared with patients who did not receive this prophylactic treatment.

What the paper's own numbers show

Method. A 95% CI for a hazard ratio, whether Wald-type from a Cox model or from a log-rank-based estimate, is built on the log scale: log HR ± 1.96 × SE. The estimate therefore lies inside the interval, at its geometric centre √(lower × upper). I computed that centre for each of the four reported intervals and compared it with the printed HR. I also took the SE of log HR implied by each interval's width, used it to recompute a two-sided P value, and checked the event percentages against the counts in Table 1. The paper does not say how the HRs were estimated: the Figure 1 legend names only the log-rank test. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
HR, symptomatic COVID-19 Text, para. 3 0.011 (95% CI 0.063–0.198) outside its CI; CI centre = 0.1117
Printed HR ÷ CI centre, symptomatic — — 0.098 (the centre is 10.2 × the printed HR)
Rounding check, symptomatic — 0.011 vs lower bound 0.063 largest value shown as 0.011 is 0.0115; smallest shown as 0.063 is 0.0625 (5.4× apart)
Distance of 0.011 below the lower bound — — 1.75 log units; the CI half-width is 0.57 log units
95% CI that HR 0.011 would have with the same SE — — 0.0062–0.0195
HR, hospitalization Text, para. 3 0.046 (0.013–0.158) inside; centre 0.0453; ratio 1.015
HR, ICU hospitalization Text, para. 3 0.045 (0.008–0.240) inside; centre 0.0438; ratio 1.027
HR, death Text, para. 3 0.076 (0.005–1.161) inside; centre 0.0762; ratio 0.997
P, symptomatic Text, para. 3 < 0.001 6.2 × 10⁻¹⁴ from the CI (z = −7.50); < 0.001 either way
P, hospitalization / ICU Text, para. 3 < 0.001 / < 0.001 1.2 × 10⁻⁶ / 0.00031
P, death Text, para. 3; Figure 1d panel 0.066 (text); "Log-rank: P = 0.064" (figure) 0.064
Symptomatic COVID-19, treated / untreated Table 1; text 41 (12.3%) / 42 (43.3%) 41/333 = 12.31%; 42/97 = 43.30%
Hospitalization Table 1; text 4 (1.2) / 11 (11.3) 1.20% / 11.34%
ICU hospitalization Table 1; text 2 (0.6) / 6 (6.2) 0.60% / 6.19%
Death Table 1; text 1 (0.3) / 2 (2) 0.30% / 2.06%
Crude risk ratio, symptomatic from Table 1 — 0.284
HR ÷ crude risk ratio from text and Table 1 — hospitalization 0.43, ICU 0.46, death 0.52; symptomatic 0.39 at the CI centre, 0.039 for the printed HR
Hazard reduction, symptomatic — 99% implied by HR 0.011 89% at the CI centre; 80% to 94% across the CI

The counts and percentages are all correct, and the hospitalization, ICU and death HRs agree with their intervals to within the three-decimal rounding of the printed bounds. Only the symptomatic HR disagrees with its interval. Its interval width (SE of log HR 0.292) is the narrowest of the four, as expected for the outcome with the most events (83). A scale check that does not rely on the CI points the same way. For the three consistent outcomes, the HR is 0.43 to 0.52 times the crude risk ratio. At 0.112 the symptomatic HR would be 0.39 times its crude risk ratio, in line with the others. At 0.011 it would be 0.039 times.

Figure

Dot-and-interval plot on a log scale: the printed symptomatic COVID-19 HR of 0.011 lies far to the left of its reported 95% CI, while the other three HRs sit at the centre of their intervals

Figure 1. The four hazard ratios reported in the letter, on one log-scaled axis. Orange dots are the HRs printed in the text and orange bars are the 95% CIs printed with them. Open blue diamonds mark the geometric centre of each printed CI, recomputed by recompute.py.txt. That centre is where a log-scale interval places its own estimate. The right-hand column gives the printed HR divided by that centre. For hospitalization, ICU hospitalization and death, the dot and diamond coincide (ratio 1.00 to 1.03). For symptomatic COVID-19 (shaded row), the printed 0.011 lies outside its interval and is 0.10 times the centre of 0.112. The thin vertical line marks HR = 1 (no difference). All values come from the paper's text or from recompute.py.txt.

Why it matters

Explanations considered

Status

Suggested correction

In the third paragraph, the HR for symptomatic COVID-19 should be checked against the original analysis output and the sentence corrected to read

(hazard ratio [HR], 0.11x; 95% confidence interval [CI], 0.063–0.198; P < 0.001; Figure 1a)

where 0.11x is the estimate from the model that produced the interval. The interval implies a value close to 0.112. If the interval is the misprinted part instead, both bounds should be corrected. The authors may also wish to state the method used to estimate the HRs, since the Figure 1 legend names only the log-rank test. They could also make the P value for death the same in the text (0.066) and in Figure 1d (0.064).