β Back to the plain-language case
Reported P < 0.0001 for a light-versus-dark coiling difference is inconsistent with the paper's counts
Sci Adv (2020) Β· doi:10.1126/sciadv.abb8930 Β· PMC10662364
Summary
The Results report that offspring of a single N. pachyderma parent coiled sinistrally more often in the 12-hour light flask than in the 24-hour dark flask: 87.5% (n = 9) against 50% (n = 12), with P < 0.0001 by a proportion z test. The counts that fit the paper's own percentages and totals are 7 of 8 and 6 of 12. They add up to the stated 13 of 20 sinistral offspring, and no count out of 9 gives 87.5%. A two-proportion z test on these counts gives z = 1.72 and two-sided P = 0.085, and Fisher's exact test gives P = 0.158. None of the standard variants I tried comes close to 0.0001. The Discussion relies on this contrast when it suggests that light conditions, or the pH and O2 changes that come with them, could contribute to coiling direction. That inference loses its statistical support, but the paper's main observations of asexual reproduction and morphological variability among clones do not depend on it.
What the paper reports
Results, "Morphology of parent and offspring", second paragraph:
Coiling direction was ascertained in 20 offspring, 13 of which coiled sinistrally (65%). There were differences in coiling direction proportion between the 12-hour light and 24-hour dark treatments (see Materials and Methods): 87.5% were sinistral (n = 9) in the 12-hour light treatment (flask βOβ), and 50% (n = 12) in the 24-hour dark treatment (flask βDβ) (P < 0.0001, by a proportion z test).
Materials and Methods, third paragraph:
No shells we recovered from flask βA,β eight offspring and the parent were recovered from flask βO,β and 17 offspring from flask βD.β
Discussion, "Phenotypic variation in a genetically similar population", second paragraph:
The relative abundance of shells coiling in the nondominant direction was greater in the 24-hour dark treatment (50%) than in the 12-hour light treatment (12.5%), indicating that factors such as light conditions or associated changes in pH and O2 could contribute to coiling direction.
What the paper's own numbers show
Method. I first found the whole-number counts consistent with each printed percentage, its printed n, the Methods recovery counts and the stated total of 13 sinistral out of 20. I then ran the two-group test the paper names (a two-proportion z test with pooled standard error, two-sided) on those counts, along with the common variants: unpooled standard error, continuity correction, one-sided, Fisher's exact test, Boschloo's exact test, and the same tests with the sinistral parent counted as a ninth shell in flask O. Last, I searched for any calculation that yields P < 0.0001. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Flask O, sinistral | Results | 87.5% (n = 9) | No count out of 9 gives 87.5% (87.5% Γ 9 = 7.875; 7/9 = 77.8%, 8/9 = 88.9%). 7/8 = 87.5%, and Methods report 8 offspring (plus the parent) from flask O |
| Flask D, sinistral | Results | 50% (n = 12) | 6/12 = 50%; 12 of the 17 offspring recovered from flask D |
| All offspring scored | Results | 13 of 20 sinistral (65%) | 7 + 6 = 13, 8 + 12 = 20, 13/20 = 65%. With n = 9 the total would be 21 |
| Nondominant (dextral) coiling | Discussion | 12.5% vs 50% | 1/8 = 12.5%, 6/12 = 50%: the same contrast as above |
| Pooled two-proportion z test, 7/8 vs 6/12 | Results | P < 0.0001 | pooled p = 0.65, SE = 0.2177, z = 1.723, two-sided P = 0.085 (one-sided 0.043) |
| Same, continuity corrected | Results | P < 0.0001 | z = 1.244, P = 0.213 (equal to chi-square with Yates correction) |
| Unpooled (Wald) z test | Results | P < 0.0001 | z = 2.019, P = 0.044 |
| Fisher's exact test | Results | P < 0.0001 | P = 0.158 (odds ratio 7.0) |
| Boschloo's exact test | Results | P < 0.0001 | P = 0.119 |
| Parent counted, 8/9 vs 6/12 | Results | P < 0.0001 | pooled z = 1.871, P = 0.061; unpooled z = 2.181, P = 0.029; Fisher P = 0.159 |
| 95% intervals | (derived) | β | Wilson: flask O 52.9% to 97.8%, flask D 25.4% to 74.6%; Newcombe for the 37.5-point difference: β5.0 to 64.2 points, which includes 0 |
| Equal group size needed for P < 0.0001 at 87.5% vs 50% | (derived) | n = 9 and n = 12 | 47 per group (pooled z test) |
| One-sample z test, flask D against flask O's estimate | (candidate explanation) | P < 0.0001 | 6/12 against a fixed 0.875: z = β3.93, P = 8.6e-5 (exact binomial P = 0.0018), but this is not a two-group comparison |
| Percentages entered as counts out of 100 | (candidate explanation) | P < 0.0001 | 87.5/100 vs 50/100: z = 5.72, P = 1.1e-8, but this is not a valid test of these data |
Figure
Figure 1. Two-sided P values for the light-versus-dark comparison of sinistral coiling, on a logarithmic axis. The orange bar and arrow mark the paper's reported value, P < 0.0001, which is an upper bound. The filled blue dots are P values recomputed from the counts that fit the paper's percentages and totals (7 of 8 versus 6 of 12, or 8 of 9 if the sinistral parent is counted). They range from 0.044 to 0.213 and lie near or above the dashed 0.05 line, more than 400 times the reported bound. The two hollow blue dots are the only calculations found that give a value below 0.0001: a one-sample z test of flask D's 6 of 12 against flask O's 87.5% treated as a fixed value (P = 8.6e-5), and the two percentages treated as counts out of 100 (P = 1.1e-8). They are shown only for comparison; neither is a valid two-group comparison of these data. All values come from recompute.py.txt.
Why it matters
The P value attaches to a secondary comparison, and the direction of the observed difference stays as reported: 1 of 8 dextral shells in the light flask against 6 of 12 in the dark flask. What changes is the strength of the evidence. As printed, P < 0.0001 tells the reader that a difference this large is very unlikely to arise by chance between two groups that coil alike. With 8 and 12 shells, the correct pooled z test gives P = 0.085, and the 95% interval for the difference runs from about β5 to 64 percentage points. The data are compatible with no effect of the treatment on coiling.
The Discussion uses this contrast to suggest that "light conditions or associated changes in pH and O2 could contribute to coiling direction". That is presented as a possibility, but readers may take the printed P < 0.0001 as strong support for it. The paper notes that the two species are "generally distinguished by shell coiling direction", so a claimed environmental effect on coiling bears on how field specimens are assigned to species. The comparison involves one parent's clonal offspring in two flasks, so even a significant result would not separate light from any other difference between the flasks, and the recomputed P values (0.03 to 0.21, two-sided) leave it without strong statistical support.
The paper's headline findings do not rest on this test: that the parent reproduced asexually, that its clonal offspring spanned four morphotypes, and that offspring coiled in both directions (13 of 20 sinistral). The title and abstract do not report the light-versus-dark P value, and the paper has no tables.
Explanations considered
- Typing error in n. The printed n = 9 is impossible for 87.5% and inconsistent with the total of 20. Methods report "eight offspring and the parent" from flask O, so n = 9 may count the parent while 87.5% uses the 8 offspring. This explains the denominator but not the P value: with either 8 or 9 shells, the tests give P between 0.029 and 0.213.
- A different variant of the z test. Pooled or unpooled standard error, with or without continuity correction, one- or two-sided: the smallest P across all these, with either denominator, is 0.0146 (one-sided, unpooled, parent counted). None reaches 0.0001.
- A different two-group test. Fisher's exact test (P = 0.158), Boschloo's exact test (P = 0.119) and chi-square with Yates correction (P = 0.213) give larger P values than the z test. Chi-square without correction is the same test as the pooled z test and gives the same P = 0.085.
- More shells than the text states. P < 0.0001 at these percentages would need about 47 shells per group. The Results say coiling was ascertained in 20 offspring, and Methods report only 8 offspring from flask O, so the scored samples cannot be that large.
- Comparison against a fixed reference value. Testing flask D's 6 of 12 against a fixed proportion of 0.875 with a one-sample z test gives z = β3.93 and P = 8.6e-5, which would be reported as P < 0.0001 and fits the wording "by a proportion z test". That treats flask O's estimate, based on 8 shells, as a known constant; the exact binomial version gives P = 0.0018. Testing flask O's 7 of 8 against 0.5 gives one-sample z test P = 0.034 (exact binomial P = 0.070). Neither is the two-group comparison the text describes.
- Comparison with the population rate of nondominant coiling. The Results state that natural populations contain 1 to 3% nondominant coilers. Against 3%, flask D's 6 of 12 (one-sided exact P = 5.8e-7) and all offspring's 7 of 20 (P = 1.2e-6) are significant, but flask O's 1 of 8 is not (P = 0.22). This is a different question from the one the sentence reports, which is the difference between treatments.
- Percentages entered as counts. A two-proportion z test on 87.5 of 100 versus 50 of 100 gives z = 5.72 and P = 1.1e-8, which would be reported as P < 0.0001. It is not a valid test, because the sample sizes are 8 and 12, not 100.
- Which calculation, if any. Two calculations found give P < 0.0001: the one-sample z test of flask D against flask O's estimate (P = 8.6e-5; exact binomial P = 0.0018) and the percentages read as counts out of 100 (P = 1.1e-8). Neither is a valid comparison of the two flasks, and no two-group test on the paper's counts comes close. The source of the printed value cannot be established from the paper.
- Conversion or typesetting artifact. Ruled out (see Status): the publisher's typeset PDF, the Europe PMC XML and the PMC text all print the same digits, with a real less-than sign and no superscripts or minus signs.
Status
As of 2026-10-23:
- Version of record. The doi.org and science.org pages were not reachable by script (HTTP 403), so the text was checked in the publisher's typeset PDF as mirrored on NSF PAR (par.nsf.gov/servlets/purl/10197656; 8 pages, header "Sci. Adv. 2020; 6 : eabb8930 10 July 2020", published 10 July 2020), in the Europe PMC full-text XML, and in the PMC archived text. All three carry the same sentences and numbers quoted above.
- Supplement. The Europe PMC supplementary archive holds abb8930_SM.pdf (captions for Movies S1 to S3 only) and three movie files. It contains no counts per flask and no statistics. The data statement says additional data may be requested from the authors. The supplement does not resolve the discrepancy.
- Corrections and notices. No correction, erratum, expression of concern or retraction was found in Crossref (no update-to, updated-by or relation entries), Crossref Labs (which carries Retraction Watch data), PubMed Central or Europe PMC.
- PubPeer. The PubPeer publication lookup for the DOI returned no comments, and a web search found no PubPeer thread.
Suggested correction
In Results, "Morphology of parent and offspring", second paragraph, give the flask O denominator that matches 87.5% (n = 8 offspring), or, if n = 9 includes the parent, state that and give the matching percentage (88.9%). Replace "P < 0.0001, by a proportion z test" with the value for the counts actually compared, for example "7 of 8 versus 6 of 12; P = 0.085, two-sided two-proportion z test (Fisher's exact test, P = 0.16)", and state that the difference is not statistically significant. In the Discussion, "Phenotypic variation in a genetically similar population", second paragraph, the sentence on light conditions and coiling direction could be reworded to present the 12.5% versus 50% contrast as a small-sample observation from two flasks that was not statistically significant.