Paperonium

← Back to the plain-language case

The abstract's two interval cancer rates each pair an estimate with a mismatched interval

JAMA Netw Open (2022) · doi:10.1001/jamanetworkopen.2022.44212 · PMC9716401

Summary

The Abstract and Results give the pooled interval cancer rate, the paper's secondary outcome, as 3.73 per 1000 examinations in both groups: with a 95% CI of 0.84-3.98 after mastectomy without reconstruction, and 0.41-2.73 after mastectomy with reconstruction. The second estimate lies outside its own interval, and that interval is the one printed for the clinically occult rate in the same group (1.06; 0.41-2.73). In the first group, the printed interval, the printed odds ratio and the paper's own Figure 3 all put the estimate near 1.8, not 3.73. Figure 3 plots the with-reconstruction estimate at 3.73 with an interval of about 1.8-7.8. So the text appears to have swapped two numbers, and the paper's own figure shows the values that belong there. The paper's main conclusion, which rests on clinically occult rates, is not affected. But the secondary outcome is misreported in the Abstract, and no correction has been issued.

What the paper reports

Abstract, Results:

For mastectomy without reconstruction, the rate of clinically occult (nonpalpable) cancer per 1000 examinations (2.96; 95% CI, 1.38-6.32) and the interval cancer rate per 1000 examinations (3.73; 95% CI, 0.84-3.98) were lower than the overall cancer detection rate (including both palpable and nonpalpable lesions) per 1000 examinations (6.41; 95% CI, 3.09-13.25) across all imaging modalities.

The interval cancer rate per 1000 examinations for mastectomy with reconstruction (3.73; 95% CI, 0.41-2.73) was comparable to the pooled cancer detection rate per 1000 examinations (4.73; 95% CI, 2.32-9.63) across all imaging modalities.

Results, "Cancer Detection Rates After Mastectomy With or Without Reconstruction", first paragraph:

For mastectomy without reconstruction, the pooled overall cancer detection rate per 1000 examinations was 6.41 (95% CI, 3.09-13.25), whereas the pooled rate of interval cancer per 1000 examinations was 3.73 (95% CI, 0.84-3.98), and the rate of clinically occult (nonpalpable) cancer was 2.96 (95% CI, 1.38-6.32) per 1000 examinations.

Same section, second paragraph:

The pooled interval cancer rate per 1000 examinations (3.73; 95% CI, 0.41-2.73) was comparable to the pooled cancer detection rate observed among patients who underwent mastectomy with reconstruction. The rate of clinically occult (nonpalpable) cancer per 1000 examinations was 1.06 (95% CI, 0.41-2.73).

Results, "Comparisons Among Surveillance Methods and Outcomes":

For mastectomy without reconstruction, the OR for occult cancer was 0.46 (95% CI, 0.30-0.70; P < .001) times the overall cancer detection rate, and the OR for interval cancer was 0.28 (95% CI, 0.18-0.46; P < .001) times the overall cancer detection rate.

However, the corresponding ORs were not statistically significant for mastectomy with reconstruction; the OR for occult cancer was 0.36 (95% CI, 0.08-1.67; P = .08) times the overall cancer detection rate, and the OR for interval cancer was 2.04 (95% CI, 0.53-7.87; P = .17) times the overall cancer detection rate.

Methods, "Outcomes and Definitions" and "Data Synthesis":

The secondary outcome was the pooled interval cancer rate, defined as ipsilateral cancer detected after a normal imaging procedure but before any subsequent imaging.

We used the generalized linear mixed model framework to meta-analyze the proportion of cases across studies.

Pooled estimates and 95% CIs from the model were presented as the number of cases per 1000 examinations.

Discussion:

Considering interval cancer rates, in the clinical scenario of mastectomy with reconstruction, the interval cancer rate was similar to the overall cancer detection rate, whereas this was not the case among patients who underwent mastectomy without reconstruction, where interval cancer rates were lower than overall cancer detection rates. This difference in trends can be explained by the higher cancer detection rates among patients who underwent mastectomy without reconstruction in general rather than by a true difference in the interval cancer rates between the 2 clinical scenarios.

Figure 3 is titled "Pooled Cancer Detection Rates, Averaged Over Imaging Modality, Among Patients After Mastectomy Without or With Reconstruction". It plots the interval, clinically occult and overall rates for each group as markers with 95% CI whiskers on a linear axis from 0 to 20.

What the paper's own numbers show

Method. I used three checks, all based only on numbers the paper prints or plots.

  1. Interval centre. The rates are pooled with a binomial generalized linear mixed model (GLMM). Its 95% CIs are symmetric on the logit scale. At rates of a few per 1000, that makes them almost exactly symmetric on the log scale, so the estimate should sit at the CI's geometric midpoint, √(lo × hi). For the seven printed rates other than the two interval rates, this rule recovers the printed estimate to within 0.03 per 1000. On the log scale every one of them sits within 0.004 half-widths of its CI centre.
  2. Odds-ratio cross-check. Multiplying the odds of the overall rate by a printed odds ratio gives the rate that odds ratio implies. Without reconstruction, this reproduces the printed occult rate exactly (0.46 with 6.41 gives 2.96).
  3. Figure 3. I measured the plotted marker centres and whisker ends from the pixels of Figure 3 as archived in PMC. The axis was calibrated by a linear fit to the ticks at 0, 5, 10, 15 and 20; one pixel is 0.095 per 1000. For the four occult and overall rates (two per group), the measured values agree with the printed text to within about one pixel (at most 0.07).

Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
Interval rate, with reconstruction: is the estimate inside its CI? Abstract; Results p2 3.73 (0.41-2.73) No. 3.73 is 1.00 above the upper limit; rounding can move each value by at most 0.005
Centre of that CI, √(0.41 × 2.73) Abstract; Results p2 n/a 1.06, which equals the printed occult rate for the same group, 1.06 (0.41-2.73)
Interval rate, with reconstruction, Figure 3 Figure 3 plotted 3.73 (1.78-7.78); the centre of that whisker is 3.72
Interval rate, without reconstruction: centre of CI √(0.84 × 3.98) Abstract; Results p1 3.73 (0.84-3.98) 1.83. 3.73 sits 0.92 half-widths (log scale) above the centre, next to the upper limit
Same rate implied by OR 0.28 × overall rate 6.41 Results, "Comparisons" OR 0.28 (0.18-0.46) 1.80
Same rate, Figure 3 Figure 3 plotted 1.83 (0.83-3.97)
Occult rate, without reconstruction Abstract; Results p1 2.96 (1.38-6.32) CI centre 2.95; OR 0.46 × 6.41 gives 2.96; Figure 3 shows 2.93 (1.40-6.35)
Overall rate, without reconstruction Abstract; Results p1 6.41 (3.09-13.25) CI centre 6.40; Figure 3 shows 6.36 (3.02-13.21)
Occult rate, with reconstruction Results p2; Additional Analyses 1.06 (0.41-2.73) CI centre 1.06; Figure 3 shows 1.07 (0.35-2.73)
Overall rate, with reconstruction Abstract; Results p2 4.73 (2.32-9.63) CI centre 4.73; Figure 3 shows 4.74 (2.35-9.69)
Overall rate by modality (mammography, ultrasonography, MRI) Abstract; Results 1.86 (1.05-3.30); 2.66 (1.48-4.76); 5.17 (1.49-17.75) CI centres 1.86, 2.65, 5.14 (logit-scale centres 1.86, 2.66, 5.17)

The two interval rates are the only printed rates that fail the centre check. Each fails in a different way. With reconstruction, the estimate 3.73 is confirmed by Figure 3, but the printed interval is the occult-rate interval. Without reconstruction, the printed interval is confirmed by Figure 3 (to within one pixel), but the estimate should be about 1.8. Three independent routes give that value: 1.83, 1.80 and 1.83. The value 3.73 printed for this group is the with-reconstruction estimate.

Figure

The printed interval cancer rates against the values plotted in the paper's own Figure 3 and recomputed from its printed numbers

Figure 1. Pooled interval cancer rates per 1000 examinations, with 95% CIs, on one linear axis. Orange circles and bars are the rates as printed in the Abstract and Results. Blue diamonds and bars are the rates as plotted in the paper's own Figure 3, measured from its pixels (resolution about 0.1). Without reconstruction (top), the printed 3.73 sits at the top of its interval. Figure 3 and two recomputations from printed numbers put it at about 1.8: the centre of the printed CI, 1.83 (open blue circle), and the printed odds ratio 0.28 applied to the printed overall rate 6.41, which gives 1.80 (open blue triangle). With reconstruction (bottom), the printed estimate of 3.73 lies outside its printed interval. That interval matches, digit for digit, the interval printed for the clinically occult rate in the same group (grey square, 1.06; dotted guides). Figure 3 plots the interval rate at 3.73 with an interval of about 1.78-7.78. The axis starts at zero and no difference is exaggerated.

Why it matters

Explanations considered

Status

Suggested correction

In the Abstract Results and in the first Results paragraph on reconstruction status, the interval cancer rate for mastectomy without reconstruction should read approximately

1.8 (95% CI, 0.84-3.98)

instead of 3.73 (95% CI, 0.84-3.98). The exact value should come from the model output; Figure 3, the CI centre and the odds ratio give 1.83, 1.83 and 1.80.

In the Abstract Results and in the second Results paragraph, the interval cancer rate for mastectomy with reconstruction should read

3.73 (95% CI, approximately 1.8-7.8)

instead of 3.73 (95% CI, 0.41-2.73), with the exact limits taken from the model output. The interval 0.41-2.73 should stay only with the clinically occult rate, 1.06.

The authors may also wish to check the with-reconstruction odds ratios (0.36 and 2.04) against the pooled rates they compare, and to confirm that the reported P values are two-sided, as the Methods state.