Paperonium

← Back to the plain-language case

Abstract calls a subgroup sex difference significant (P < .001); the Results report P = .28

JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.17149 · PMC12188365

Summary

The abstract says the female share among the additional eligible people with lung cancer under the 20-year smoking-duration (IPN-Potter) criterion, 70 of 128 (55%), differed significantly from USPSTF 2021 (55% vs 48%; P < .001). The Results section reports the same comparison as not significant: 55% vs 50% [370 of 747]; P = .28. A Pearson χ² test on the paper's own counts (70/128 vs 370/747) gives P = .281, which reproduces the Results. The abstract's "48%" equals 1847/3840, the female share of all USPSTF 2021 eligibles, and "55% vs 48%; P < .001" is word for word the result of a different comparison, the one for all additional eligibles. The error is confined to one abstract sentence. It turns a non-significant subgroup result into a significant one in the most-read part of the paper. The headline conclusions rest on other, correctly reported comparisons.

What the paper reports

Abstract, Results paragraph (the sentence in question):

The additional eligible individuals with lung cancer in IPN programs who were female included 70 of 128 (55%) by IPN-Potter criteria, 89 of 208 (43%) by IPN-ACS criteria, and 143 of 304 (47%) by IPN-Potter-ACS criteria, with statistically significant differences between the IPN-Potter criteria and USPSTF 2021 criteria (55% vs 48%; P < .001).

Results, section "Screening Eligibility in Those With Lung Cancer":

The additional eligible individuals with lung cancer in IPN (IPN-Potter Extra, IPN-ACS Extra, IPN-Potter-ACS Extra) who were female included 70 of 128 (55%) in Potter criteria, 89 of 208 (43%) in ACS criteria, and 143 of 304 (47%) in Potter-ACS criteria, respectively; not significantly different between IPN-Potter Extra and IPN-USPSTF 2021 (55% vs 50% [370 of 747]; P = .28) (Figure 2).

The same Results section, preceding paragraph, gives the comparator group:

compared with 50% (370 of 747) in IPN-USPSTF 2021

The comparison for all additional eligibles, which carries the same "55% vs 48%; P < .001", appears in the sentence just before the one in question in the abstract:

There were significant differences in the number of additional eligible individuals who were female (55% vs 48%; P < .001) and Black (35% vs 22%; P < .001) between the IPN-Potter and USPSTF 2021 criteria.

and in Results, section "Additional Persons Eligible":

with significant differences in the numbers of females included between the IPN-Potter Extra and IPN-USPSTF 2021 criteria (55% vs 48%; P < .001) (Figure 2)

The test, from Methods, "Statistical Analysis" (the abstract also names the χ2 test):

using an α = .05 to determine statistical significance without adjusting for multiple comparisons. Tests were 2-sided.

What the paper's own numbers show

Method. I took the counts printed in the text and confirmed them against eTables 1 and 2 of Supplement 1. eTable 2 gives Female 370 (49.53%) of 747 for IPN-USPSTF2021 and 70 (54.69%) of 128 for IPN-Potter Extra. eTable 1 gives 1847 (48.1%) of 3840 and 606 (54.94%) of 1103. I then ran the 2-sided Pearson χ² test that Methods describes on each 2 × 2 table. As checks I also ran the Yates-corrected χ², Fisher's exact test and a pooled two-proportion z test. Proportions carry 95% Wilson intervals. recompute.py.txt prints every number below.

Quantity Location Reported Recomputed from the paper's own numbers
Female, IPN-Potter Extra with lung cancer Abstract; Results 70 of 128 (55%) 54.7% (95% CI 46.1–63.0)
Female, IPN-USPSTF 2021 with lung cancer Results 50% (370 of 747) 49.5% (95% CI 46.0–53.1)
Comparator % in the abstract's lung-cancer sentence Abstract 48% 48% is 1847/3840 = 48.1%, the female share of all IPN-USPSTF 2021 eligibles, not of those with lung cancer (49.5%)
P, 70/128 vs 370/747 Results P = .28 P = .281 (χ² = 1.16; z = 1.08). Yates P = .326; Fisher P = .294
P, the same comparison Abstract P < .001 P = .281
P, 70/128 vs 1847/3840 (if the 48% group was meant) Abstract, alternative reading P < .001 P = .142 (Yates .168; Fisher .151)
Female, all additional eligibles: 606/1103 vs 1847/3840 Abstract, previous sentence; Results "Additional Persons Eligible" 55% vs 48%; P < .001 54.9% vs 48.1%; P = 6.2 × 10⁻⁵ (consistent)
Black, lung-cancer subgroup: 50/128 vs 162/747 Abstract; Results 39% vs 22%; P < .001 39.1% vs 21.7%; P = 2.2 × 10⁻⁵ (consistent)
Black, all additional eligibles: 382/1103 vs 777/3840 Abstract (35% vs 22%); Results (35% vs 20%) P < .001 34.6% vs 20.2%; P = 2.6 × 10⁻²³. The abstract's 22% matches 162/747 (21.7%), not 777/3840

The Results value reproduces to the second decimal place from the printed counts with the test the Methods name. With these counts, P < .001 against 70/128 would require the USPSTF 2021 lung-cancer group to be at most 292/747 (39.1%) female, not 370/747.

Figure

The abstract and the Results give opposite significance verdicts for the same comparison; the paper's counts reproduce the Results

Figure 1. The two passages, shortened to their key clauses (full quotes above). Both describe the female share among additional eligible people with lung cancer under IPN-Potter (70 of 128) compared with IPN-USPSTF 2021. The values the paper reports are in orange. The blue box gives values recomputed from the paper's own counts with the 2-sided Pearson χ² test named in Methods. The counts reproduce the Results (P = .28), not the abstract (P < .001). The abstract's comparator (48%) is the female share of all USPSTF 2021 eligibles. "55% vs 48%; P < .001" is the correct result for all additional eligibles (606/1103 vs 1847/3840), reported in the sentence before.

Female share in the three groups involved, with 95% intervals

Figure 2. Female share in each group. Orange diamonds are the rounded percentages the paper reports. Blue dots and lines are proportions recomputed from the printed counts, with 95% Wilson intervals. The 128 additional eligibles with lung cancer form a small group, and their interval (46.1–63.0%) overlaps both possible comparators. The difference from the USPSTF 2021 lung-cancer group is 5.2 percentage points (P = .28). Even against the abstract's 48% group it is 6.6 points (P = .14). The axis starts at 35% to show the intervals. The figure is a dot-and-interval plot with no bars, so the cut-off axis does not change how large any difference looks relative to its interval.

Why it matters

Explanations considered

Status

Suggested correction

In the Results paragraph of the abstract, replace

with statistically significant differences between the IPN-Potter criteria and USPSTF 2021 criteria (55% vs 48%; P < .001).

with

with no statistically significant difference between the IPN-Potter criteria and USPSTF 2021 criteria (55% vs 50%; P = .28).

so that it agrees with the Results section and with eTable 2. In the sentence before it, the authors may also wish to change the USPSTF 2021 comparator for Black individuals, "(35% vs 22%; P < .001)", to "(35% vs 20%; P < .001)", matching the Results (777 of 3840).