Paperonium

← Back to the plain-language case

Reported P < 0.0001 for a light-versus-dark coiling difference is inconsistent with the paper's counts

Sci Adv (2020) Β· doi:10.1126/sciadv.abb8930 Β· PMC10662364

Summary

The Results report that offspring of a single N. pachyderma parent coiled sinistrally more often in the 12-hour light flask than in the 24-hour dark flask: 87.5% (n = 9) against 50% (n = 12), with P < 0.0001 by a proportion z test. The counts that fit the paper's own percentages and totals are 7 of 8 and 6 of 12. They add up to the stated 13 of 20 sinistral offspring, and no count out of 9 gives 87.5%. A two-proportion z test on these counts gives z = 1.72 and two-sided P = 0.085, and Fisher's exact test gives P = 0.158. None of the standard variants I tried comes close to 0.0001. The Discussion relies on this contrast when it suggests that light conditions, or the pH and O2 changes that come with them, could contribute to coiling direction. That inference loses its statistical support, but the paper's main observations of asexual reproduction and morphological variability among clones do not depend on it.

What the paper reports

Results, "Morphology of parent and offspring", second paragraph:

Coiling direction was ascertained in 20 offspring, 13 of which coiled sinistrally (65%). There were differences in coiling direction proportion between the 12-hour light and 24-hour dark treatments (see Materials and Methods): 87.5% were sinistral (n = 9) in the 12-hour light treatment (flask β€œO”), and 50% (n = 12) in the 24-hour dark treatment (flask β€œD”) (P < 0.0001, by a proportion z test).

Materials and Methods, third paragraph:

No shells we recovered from flask β€œA,” eight offspring and the parent were recovered from flask β€œO,” and 17 offspring from flask β€œD.”

Discussion, "Phenotypic variation in a genetically similar population", second paragraph:

The relative abundance of shells coiling in the nondominant direction was greater in the 24-hour dark treatment (50%) than in the 12-hour light treatment (12.5%), indicating that factors such as light conditions or associated changes in pH and O2 could contribute to coiling direction.

What the paper's own numbers show

Method. I first found the whole-number counts consistent with each printed percentage, its printed n, the Methods recovery counts and the stated total of 13 sinistral out of 20. I then ran the two-group test the paper names (a two-proportion z test with pooled standard error, two-sided) on those counts, along with the common variants: unpooled standard error, continuity correction, one-sided, Fisher's exact test, Boschloo's exact test, and the same tests with the sinistral parent counted as a ninth shell in flask O. Last, I searched for any calculation that yields P < 0.0001. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Flask O, sinistral Results 87.5% (n = 9) No count out of 9 gives 87.5% (87.5% Γ— 9 = 7.875; 7/9 = 77.8%, 8/9 = 88.9%). 7/8 = 87.5%, and Methods report 8 offspring (plus the parent) from flask O
Flask D, sinistral Results 50% (n = 12) 6/12 = 50%; 12 of the 17 offspring recovered from flask D
All offspring scored Results 13 of 20 sinistral (65%) 7 + 6 = 13, 8 + 12 = 20, 13/20 = 65%. With n = 9 the total would be 21
Nondominant (dextral) coiling Discussion 12.5% vs 50% 1/8 = 12.5%, 6/12 = 50%: the same contrast as above
Pooled two-proportion z test, 7/8 vs 6/12 Results P < 0.0001 pooled p = 0.65, SE = 0.2177, z = 1.723, two-sided P = 0.085 (one-sided 0.043)
Same, continuity corrected Results P < 0.0001 z = 1.244, P = 0.213 (equal to chi-square with Yates correction)
Unpooled (Wald) z test Results P < 0.0001 z = 2.019, P = 0.044
Fisher's exact test Results P < 0.0001 P = 0.158 (odds ratio 7.0)
Boschloo's exact test Results P < 0.0001 P = 0.119
Parent counted, 8/9 vs 6/12 Results P < 0.0001 pooled z = 1.871, P = 0.061; unpooled z = 2.181, P = 0.029; Fisher P = 0.159
95% intervals (derived) β€” Wilson: flask O 52.9% to 97.8%, flask D 25.4% to 74.6%; Newcombe for the 37.5-point difference: βˆ’5.0 to 64.2 points, which includes 0
Equal group size needed for P < 0.0001 at 87.5% vs 50% (derived) n = 9 and n = 12 47 per group (pooled z test)
One-sample z test, flask D against flask O's estimate (candidate explanation) P < 0.0001 6/12 against a fixed 0.875: z = βˆ’3.93, P = 8.6e-5 (exact binomial P = 0.0018), but this is not a two-group comparison
Percentages entered as counts out of 100 (candidate explanation) P < 0.0001 87.5/100 vs 50/100: z = 5.72, P = 1.1e-8, but this is not a valid test of these data

Figure

Two-sided P values for flask O versus flask D on a log axis: the reported P < 0.0001 against values recomputed from 7/8 and 6/12 sinistral offspring

Figure 1. Two-sided P values for the light-versus-dark comparison of sinistral coiling, on a logarithmic axis. The orange bar and arrow mark the paper's reported value, P < 0.0001, which is an upper bound. The filled blue dots are P values recomputed from the counts that fit the paper's percentages and totals (7 of 8 versus 6 of 12, or 8 of 9 if the sinistral parent is counted). They range from 0.044 to 0.213 and lie near or above the dashed 0.05 line, more than 400 times the reported bound. The two hollow blue dots are the only calculations found that give a value below 0.0001: a one-sample z test of flask D's 6 of 12 against flask O's 87.5% treated as a fixed value (P = 8.6e-5), and the two percentages treated as counts out of 100 (P = 1.1e-8). They are shown only for comparison; neither is a valid two-group comparison of these data. All values come from recompute.py.txt.

Why it matters

The P value attaches to a secondary comparison, and the direction of the observed difference stays as reported: 1 of 8 dextral shells in the light flask against 6 of 12 in the dark flask. What changes is the strength of the evidence. As printed, P < 0.0001 tells the reader that a difference this large is very unlikely to arise by chance between two groups that coil alike. With 8 and 12 shells, the correct pooled z test gives P = 0.085, and the 95% interval for the difference runs from about βˆ’5 to 64 percentage points. The data are compatible with no effect of the treatment on coiling.

The Discussion uses this contrast to suggest that "light conditions or associated changes in pH and O2 could contribute to coiling direction". That is presented as a possibility, but readers may take the printed P < 0.0001 as strong support for it. The paper notes that the two species are "generally distinguished by shell coiling direction", so a claimed environmental effect on coiling bears on how field specimens are assigned to species. The comparison involves one parent's clonal offspring in two flasks, so even a significant result would not separate light from any other difference between the flasks, and the recomputed P values (0.03 to 0.21, two-sided) leave it without strong statistical support.

The paper's headline findings do not rest on this test: that the parent reproduced asexually, that its clonal offspring spanned four morphotypes, and that offspring coiled in both directions (13 of 20 sinistral). The title and abstract do not report the light-versus-dark P value, and the paper has no tables.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In Results, "Morphology of parent and offspring", second paragraph, give the flask O denominator that matches 87.5% (n = 8 offspring), or, if n = 9 includes the parent, state that and give the matching percentage (88.9%). Replace "P < 0.0001, by a proportion z test" with the value for the counts actually compared, for example "7 of 8 versus 6 of 12; P = 0.085, two-sided two-proportion z test (Fisher's exact test, P = 0.16)", and state that the difference is not statistically significant. In the Discussion, "Phenotypic variation in a genetically similar population", second paragraph, the sentence on light conditions and coiling direction could be reworded to present the 12.5% versus 50% contrast as a small-sample observation from two flasks that was not statistically significant.