Paperonium

โ† Back to the plain-language case

95% upper bound for zero neural tube defects among 384 dolutegravir-exposed outcomes is tenfold too small

Lancet HIV (2021) ยท doi:10.1016/S2352-3018(20)30268-X ยท PMC7869891

Summary

The paper's primary safety outcome is neural tube defect (NTD) prevalence after periconceptional dolutegravir (DTG) exposure. It observes no NTDs among 384 DTG-exposed and 1068 efavirenz (EFV)-exposed birth outcomes (Table 3). In the Abstract and the Results it reports a 95% Wilson confidence interval of 0โ€“0ยท0010 for the DTG group and 0โ€“0ยท0036 for the EFV group. For 0 events in n, the Wilson 95% upper bound is zยฒ/(n + zยฒ). With n = 384 that is 0ยท0099, about ten times the printed value; a bound of 0ยท0010 would need about 3838 outcomes, and no DTG-exposed pregnancy count in the paper comes close. The same method reproduces every other Wilson (prevalence) interval in the paper, including the EFV bound. The paper cites five DTG-exposed NTD rates from other studies: 1ยท04% (the Botswana estimate used in the Methods power calculation), 0ยท94%, 0ยท66% and 0ยท30% (Botswana) and 0ยท4% (the Antiretroviral Pregnancy Registry). The printed DTG bound (0ยท10%) lies below all five, so it appears to exclude every one of them. The bound implied by the paper's own counts (0ยท99%) contains four of them, 0ยท94%, 0ยท66%, 0ยท4% and 0ยท30%. Only the 1ยท04% power-calculation rate lies above it, by 0ยท05 percentage points.

What the paper reports

Abstract, Methods:

NTD incidences were calculated with 95% Wilson confidence intervals (95%CI).

Abstract, Findings:

There were no NTD in either DTG-exposed (0 [95%CI 0, 0ยท0010]) or EFV-exposed groups (0 [95%CI 0, 0ยท0036]).

Methods, "Variable definitions and statistical analysis":

We computed Wilson 95% confidence intervals (95%CI) for prevalence estimates of the primary outcomes.

Results, third paragraph:

The estimated NTD prevalence for DTG-exposed infants was 0 (95%CI 0-0ยท0010) and 0 (95%CI 0-0ยท0036) in EFV-exposed infants. Our composite outcome was more frequent in DTG-exposed fetuses (n=25, 7% [95%CI 0ยท044-0ยท094]) compared to those EFV-exposed (n=43, 4% [95%CI 0ยท030-0ยท054])

Table 3 ("Birth outcomes of periconceptionally DTG- and EFV- exposed women, Brazil, 2015-2018"), excerpt (selected rows; the P-value column and the "Outcome" row that carries its P value are omitted):

| Only EFV regimen(N=1068) | Any DTG regimen(N=384) |

Neural tube defects, n (%) | 0 (0) | 0 (0)

Live birth | 1025 (96) | 359 (93)

Stillbirth | 15 (1) | 2 (1)

Abortion | 28 (3) | 23 (6)

Results, "Updated NTD incidence estimate":

2 / (0ยท78 * 900 + 382) = 0ยท0018 or 1ยท8 per 1,000 DTG-exposed pregnancies (95%CI 0ยท5-6ยท7 per 1,000 DTG-exposed pregnancies)

Research in context panel, "Evidence before this study" (the comparison rates):

out of 426 women exposed to DTG at conception, four gave birth to infants with NTD (0ยท94%) compared to 0ยท10% among those exposed to other antiretrovirals. Since that initial warning, updated data from the Botswana study showed a decreased prevalence compared to the initial evaluation though statistically increased compared to women on non-DTG regimens (0ยท30% vs 0ยท10%). Other recent smaller studies from Botswana demonstrated increased NTD prevalence among women with DTG exposure (1 among 152 women or 0ยท66%) but statistical analyses comparing risk to other groups included the null. Lastly, data from North America and Europe using the Antiretroviral Pregnancy registry observed 1 NTD event among 248 DTG-exposed women (0ยท4%).

Methods, "Study design and data sources" (the power calculation):

Assuming a NTD prevalence of 0ยท06% for births among DTG-unexposed women and a prevalence of 1ยท04% in the DTG-exposed women (the estimated prevalence from Botswana available at the time)

Abstract, Interpretation:

Neither DTG or EFV exposure was associated with NTD in our national cohort; incidence of NTD is likely well under 1% among DTG-exposed HIV-positive women

The version of record repeats the bound. The publisher's edited abstract, as indexed in PubMed (PMID 33387477), reads:

There were no NTDs in either dolutegravir-exposed (0, 95% CI 0-0ยท0010) or efavirenz-exposed groups (0, 95% CI 0-0ยท0036).

What the paper's own numbers show

Method. I computed the Wilson score interval at the stated 95% level (z = 1ยท959964) for 0 events, using every denominator the paper gives for each group: birth outcomes (Table 3 N), women, and live births. For x = 0 the upper bound has the closed form zยฒ/(n + zยฒ). I inverted that formula to find the n at which the bound equals 0ยท0010. To rule out a different method or level, I also computed Clopperโ€“Pearson, the rule of three, and Wilson bounds at 90% (equal to a one-sided 95% bound) and 99%. To test whether the authors' method and denominators are as stated, I recomputed the paper's other Wilson intervals: the composite outcome in each group and the updated DTG estimate. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
DTG NTD, 0/384 birth outcomes, Wilson 95% upper bound Abstract (Findings); Results ยถ3; Table 3 N 0ยท0010 0ยท0099 (0ยท99%), 9ยท9 times the reported value
DTG NTD, 0/382 women Abstract; Results (same 0ยท0010) 0ยท0100
DTG NTD, 0/359 live births Table 3 (same 0ยท0010) 0ยท0106
EFV NTD, 0/1068 birth outcomes, Wilson 95% upper bound Abstract (Findings); Results ยถ3; Table 3 N 0ยท0036 0ยท0036 (0ยท00358), matches
EFV NTD, 0/1045 women; 0/1025 live births Results; Table 3 (same 0ยท0036) 0ยท0037; 0ยท0037
DTG 0/384, other methods and levels derived 0ยท0010 Clopperโ€“Pearson 95% 0ยท0096; rule of three 0ยท0078; Wilson 90% (one-sided 95%) 0ยท0070; Wilson 99% 0ยท0170
n at which a 0/n Wilson 95% upper bound equals 0ยท0010 derived โ€” 3838 (the value rounds to 0ยท0010 only for n from 3655 to 4039)
Largest DTG-exposed pregnancy or outcome counts Abstract; Results; Table 3 382 women; 384 birth outcomes; updated DTG denominator 1084 (whole analysed cohort: 1427 women, 1452 outcomes) all below 3655
Larger counts in the paper that are not DTG-exposed pregnancy denominators Results ยถ1; Introduction 3390 possible pregnancies screened; 2507 selected for investigation; 22 624 Brazilian women on DTG aged 15โ€“49 0/n Wilson 95% upper bounds 0ยท0011, 0ยท0015 and 0ยท0002; none falls in the 3655โ€“4039 range that gives 0ยท0010
Composite outcome DTG, 25/384 Results ยถ3; Table 3 (2 stillbirths + 23 abortions) 7% (0ยท044โ€“0ยท094) 6ยท5% (0ยท0445โ€“0ยท0943), matches
Composite outcome EFV, 43/1068 Results ยถ3; Table 3 (15 + 28) 4% (0ยท030โ€“0ยท054) 4ยท0% (0ยท0300โ€“0ยท0538), matches
Updated DTG estimate, 2/(0ยท78 ร— 900 + 382) = 2/1084 Results, updated estimate; Abstract 0ยท0018 (0ยท0005โ€“0ยท0067) 0ยท00185 (0ยท00051โ€“0ยท00670), matches
Table 3 column totals Table 3 N = 384; N = 1068 359 + 2 + 23 = 384; 1025 + 15 + 28 = 1068

The DTG NTD bound is the only Wilson (prevalence) interval in the paper that does not reproduce. Every other Wilson interval reproduces with the stated method and the Table 3 denominators. The bootstrap 95% CIs for the propensity-weighted odds ratios (Methods; Results; Figure 2 of the paper) depend on individual-level data and were not recomputed.

Figure

Reported and recomputed 95% confidence intervals for NTD prevalence, with the comparison rates the paper cites

Figure 1. NTD prevalence intervals on one axis, in percent. Each pair of rows shows the interval as reported in the paper (orange) and as recomputed from the paper's own counts with the Wilson 95% method the paper states (blue). Dots are point estimates and the end ticks are upper bounds, with values printed next to them. For EFV (0/1068) and for the updated DTG estimate (2/1084) the reported and recomputed intervals coincide. For DTG (0/384) the recomputed upper bound is 0ยท99%, 9ยท9 times the reported 0ยท10%. The lower rows show every NTD rate the paper cites for comparison, each with a thin dashed line running up through the intervals: five DTG-exposed rates (filled diamonds: Botswana 1ยท04% from the power calculation, 0ยท94%, 0ยท66% and 0ยท30%, and the Pregnancy Registry 0ยท4%) and two rates without DTG exposure (hollow diamonds: Botswana non-DTG 0ยท10%, Brazil general population 0ยท06%). As printed, the DTG interval ends at 0ยท10% and lies below all five DTG-exposed rates. As recomputed, it contains four of them; only the 1ยท04% power-calculation rate lies above it.

Wilson 95% upper bound for zero events as a function of sample size

Figure 2. The Wilson 95% upper bound for 0 events in n, zยฒ/(n + zยฒ), on logโ€“log axes (blue line). The recomputed DTG (n = 384, 0ยท99%) and EFV (n = 1068, 0ยท36%) bounds lie on the line, and the EFV value matches the paper. The reported DTG bound of 0ยท0010 at n = 384 (orange dot, with a dashed orange line at its level) lies far off the line. The line reaches 0ยท0010 only at n = 3838 (hollow diamond), about ten times the DTG sample and larger than the whole analysed cohort (1452 birth outcomes).

Why it matters

NTD prevalence among women exposed to DTG at conception is the outcome the study was set up to measure, after the 2018 safety signal from Botswana. Its confidence interval is the cohort's own statement of how high the DTG-associated NTD risk could plausibly be. The misprinted bound is in the Abstract Findings, which is also the version of record's abstract and is indexed in PubMed, and in the Results. The correct value is printed nowhere in the paper, so a reader cannot recover it without recomputing.

The error changes what the cohort appears to exclude.

Also unaffected: the point estimates (0 NTDs in each group), the Table 3 counts, the composite-outcome intervals and regressions, and the updated estimate (0ยท18%, 95% CI 0ยท05โ€“0ยท67%). The Interpretation's "well under 1%" is supported by the updated estimate, which is computed correctly. On the cohort data alone, though, the upper bound is 0ยท99%, just under 1%. The overall conclusions survive. What is overstated, about tenfold, is the precision of the primary-outcome estimate in the Abstract and Results.

Explanations considered

Status

Suggested correction

  1. In the Abstract (Findings) and the Results (third paragraph), replace the DTG NTD interval "0โ€“0ยท0010" with the Wilson 95% interval for 0/384, 0โ€“0ยท0099 (0โ€“0ยท0100 if the 382 women are the intended denominator). Leave the EFV interval (0โ€“0ยท0036) as it is.
  2. State the denominator used for the NTD prevalence (birth outcomes, women or live births), since the paper uses all three elsewhere.
  3. Optionally, note that the corrected DTG interval contains the DTG-exposed prevalences from the Research in context panel (0ยท94%, 0ยท66%, 0ยท4% and 0ยท30%), though not the 1ยท04% assumed in the power calculation, which fits the Discussion's statement that the study was underpowered for NTDs. The updated estimate, 0ยท0018 (0ยท0005โ€“0ยท0067), needs no change.