Paperonium

← Back to the plain-language case

Hospital-stay estimate of 8.4 days lies outside its own reported 95% CI of 4.4–6.7

Kidney Int (2021) Β· doi:10.1016/j.kint.2021.02.026 Β· PMC7927648

Summary

This retrospective cohort of 152 children hospitalized with acute COVID-19 or MIS-C reports, in the Abstract, the Results and Table 4, that acute kidney injury (AKI, stages 1–3) was associated with an 8.4-day longer hospital stay, with a 95% CI of 4.4 to 6.7 and P < 0.0001, from unadjusted simple linear regression. The point estimate lies 1.7 days above the upper bound of its own interval. A regression CI is centred on its coefficient, and the stated interval is centred on 5.55. Every other row of Table 4 except one is centred on its estimate. At least one of the three printed numbers is therefore wrong, and the paper does not contain enough information to say which. The direction and statistical significance of the association survive every reconstruction considered, but the precision is overstated or the size is misstated. A second Table 4 row, for length of mechanical ventilation, is also not centred on its estimate.

What the paper reports

Abstract:

In unadjusted models, patients with acute kidney injury had 8.4 days longer hospitalizations compared to patients without acute kidney injury (95% confidence interval, 4.4-6.7).

Results, "Clinical course and outcomes", third paragraph:

In addition, patients with AKI had a significantly greater length of hospital stay of 8.4 days compared with patients without AKI (95% CI, 4.4–6.7; P < 0.0001).

Table 4, "Association of AKI with hospital outcomes", under the column header "Ξ² (95% CI)" (columns: outcome by AKI stage | Ξ² (95% CI) | P value):

Length of hospital stay, d AKI stage 1 + 2 + 3 | 8.4 (4.4 to 6.7) | <0.0001 AKI stage 2 + 3 | 5.2 (0.07 to 10.2) | 0.047 Length of PICU stay, d AKI stage 1 + 2 + 3 | 5 (0.86 to 9.1) | 0.02 AKI stage 2 + 3 | 2.3 (–3.7 to 8.3) | 0.45 Length of mechanical ventilation, d AKI stage 1 + 2 + 3 | –3.9 (–9.4 to 12.7) | 0.15 AKI stage 2 + 3 | –1.08 (–7.4 to 5.3) | 0.71

The same table gives mortality as a hazard ratio: "AKI stage 1 + 2 + 3 | 0.57 (0.03 to 11.8) | 0.72".

Methods, "Statistical analysis":

Multiple imputation was used for missing values in regression analysis. Subsequently, simple linear regression analysis was performed to assess the relationship between AKI and continuous outcomes, including PICU and hospital length of stay and length of time on mechanical ventilation.

The conclusions that draw on this result. Discussion, first paragraph:

AKI, in unadjusted models, was significantly associated with longer PICU and hospital length of stay.

Discussion, "Conclusions":

Pediatric COVID-19–related AKI, similar to reports in larger AKI studies, was associated with poor outcomes, such as increased PICU and hospital length of stay.

What the paper's own numbers show

Method. For simple linear regression the 95% CI of a coefficient is Ξ² Β± t(0.975, df) Γ— SE. It always contains Ξ² and is centred on it. A CI pooled across multiply imputed data sets with Rubin's rules has the same form, so the same holds. For each Table 4 row I checked whether the estimate lies inside its interval and computed the centre of the interval, the SE implied by its width (df = 150, from N = 152), and the P value implied by Ξ²/SE. For the hazard ratio I did the same on the log scale with a normal critical value. I then rebuilt the hospital-stay row under each assumption about which number is correct. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Hospital stay, AKI 1–3: is Ξ² inside its CI? Abstract; Results; Table 4 8.4 (4.4 to 6.7) No. 8.4 is 1.7 d above the upper bound 6.7
Hospital stay, AKI 1–3: centre of the CI Table 4 8.4 5.55 (half-width 1.15; SE from width 0.582)
Same row, if 8.4 and the lower bound 4.4 are right derived upper bound 6.7; P < 0.0001 upper bound 12.4; SE 2.024; t = 4.15; P = 5.6 Γ— 10⁻⁡ (consistent with P < 0.0001)
Same row, if 8.4 and the upper bound 6.7 are right derived β€” lower bound 2 Γ— 8.4 βˆ’ 6.7 = 10.1, above the estimate: impossible
Same row, if the CI 4.4–6.7 is right derived Ξ² = 8.4 Ξ² = 5.55; t = 9.54; P = 3.8 Γ— 10⁻¹⁷ (also < 0.0001)
SE under the 4.4–12.4 reconstruction vs SE implied by the printed width derived β€” 2.024 / 0.582 = 3.48 times larger
Hospital stay, AKI 2–3 Table 4 5.2 (0.07 to 10.2), P = 0.047 centre 5.135; P from Ξ²/SE 0.044: consistent
PICU stay, AKI 1–3 Table 4 5 (0.86 to 9.1), P = 0.02 centre 4.98; P 0.018: consistent
PICU stay, AKI 2–3 Table 4 2.3 (–3.7 to 8.3), P = 0.45 centre 2.30; P 0.45: consistent
Mechanical ventilation, AKI 1–3 Table 4 –3.9 (–9.4 to 12.7), P = 0.15 inside, but centre +1.65; arms 5.5 below and 16.6 above; P from Ξ²/SE 0.49 (0.45 if only the 11 ventilated patients in Table 3 were analysed, df = 9): inconsistent
Mechanical ventilation, AKI 2–3 Table 4 –1.08 (–7.4 to 5.3), P = 0.71 centre –1.05; P 0.74: consistent
Mortality hazard ratio, AKI 1–3 Table 4 0.57 (0.03 to 11.8), P = 0.72 geometric centre 0.595; P 0.71: consistent
Median hospital stay, without vs with AKI Table 3 acute COVID-19 2.8 vs 22.2 d; MIS-C 4.6 vs 8.2 d differences 19.4 and 3.6 d (medians by subgroup, so they cannot reproduce the pooled Ξ²)

Five of the seven Table 4 rows are centred on their estimates to within rounding, and their printed P values agree with the P implied by their intervals. The hospital-stay row for AKI stages 1–3 is the only one whose estimate falls outside its interval. The mechanical-ventilation row for AKI stages 1–3 is the only other row that is not centred on its estimate, and its printed P = 0.15 does not match the P implied by its interval.

A few smaller inconsistencies turned up during the comparison. None of them affects a conclusion:

Quantity Location Reported Recomputed
PICU stay, AKI 1–3 Results text vs Table 4 text "95% CI, 0.86–9; P = 0.025"; table "0.86 to 9.1", 0.02 same estimate, different rounding of the bound and P
Acute COVID-19 group size Table 3 vs Table 1 and Abstract Table 3 "No AKI (N = 88)", "AKI (N = 8)" 88 + 8 = 96; Table 1 gives 97 overall and No AKI N = 89
Mechanical ventilation, acute COVID-19 with AKI Table 3 2 (33.3) of N = 8 2/8 = 25.0%
Vasopressors, acute COVID-19 with AKI Table 3 5 (32) of N = 8 5/8 = 62.5%

Figure

The Table 4 day-scale estimates and intervals, with the centre of each reported interval

Figure 1. The six day-scale rows of Table 4 on one axis, with no effect (0 days) as the thin vertical line. Orange dots and bars are the reported Ξ² and 95% CI. Blue rings mark the centre of each reported interval, computed from its printed bounds; where the orange dot sits inside the blue ring the estimate is centred, as it must be for a linear-regression CI. For hospital stay with AKI stages 1–3 the estimate 8.4 lies outside the interval, 1.7 days above its upper bound, and the interval's centre is 5.55. The dashed blue row below it is one possible reconstruction, not a correction: the interval centred on 8.4 that keeps the printed lower bound 4.4 (4.4 to 12.4, P = 6 Γ— 10⁻⁡). The mechanical-ventilation row for AKI stages 1–3 contains its estimate, but its centre (+1.65) is 5.55 days from the estimate.

Why it matters

The result appears in the Abstract as one of the study's main findings, in the Results text and in Table 4, and it supports the Discussion and Conclusions statements that AKI was associated with longer hospital stays. As printed, the interval is impossible. A reader cannot tell whether the effect is about 8.4 days with a wide interval (for example 4.4 to 12.4) or about 5.6 days with the narrow interval printed. The two readings differ in effect size by almost 3 days and in standard error by a factor of about 3.5.

What does not change: every reconstruction leaves the lower bound well above zero and gives P < 0.0001, so the qualitative statement that AKI was associated with a significantly longer unadjusted hospital stay is not overturned. Table 3 medians (2.8 vs 22.2 days in acute COVID-19, 4.6 vs 8.2 days in MIS-C) and the PICU-stay result (5 days, 0.86 to 9.1) point the same way. Length of stay is a secondary outcome. The paper states that "The primary outcome of this study was the incidence rate of AKI" (11.8% overall), and that result, the albumin and white-cell odds ratios, the MIS-C systolic-dysfunction comparison and the mortality result do not depend on this row.

The mechanical-ventilation row for AKI stages 1–3 is a second, smaller problem: the printed estimate, interval and P value cannot all come from one symmetric interval. It is reported as non-significant, and every value implied by its interval is non-significant too, so the paper's statement that there was no significant association with length of mechanical ventilation is not affected.

Explanations considered

Status

Suggested correction

  1. Re-run or re-extract the unadjusted linear regression of hospital length of stay on AKI (stages 1–3) and correct the estimate, the 95% CI or both in the Abstract, the Results text ("Clinical course and outcomes") and Table 4, together with the P value if it changes.
  2. Check the mechanical-ventilation row for AKI stages 1–3 in Table 4, whose estimate (–3.9), interval (–9.4 to 12.7) and P value (0.15) are mutually inconsistent, and state how many patients each length-of-ventilation model included.
  3. Minor: reconcile the PICU-stay interval and P value between the Results text and Table 4; correct the acute COVID-19 No AKI column size in Table 3 (88 vs 89); and check the percentages for mechanical ventilation (2 of 8) and vasopressors (5 of 8) in the acute COVID-19 AKI column of Table 3.