Paperonium

← Back to the plain-language case

Reported Fisher's exact p < 0.01 for 18/20 versus 3/6 patent grafts recomputes to 0.062

Nat Commun (2025) · doi:10.1038/s41467-025-68075-8 · PMC12868887

Summary

In the rabbit carotid model, the Results compare 2-week graft patency between the new micro-nano structured grafts (MnVGs, 18 of 20 patent) and electrospinning-only control grafts (3 of 6 patent), and give "p < 0.01, Fisher’s exact test". Fisher's exact test on these counts gives p = 0.062, two-sided and one-sided alike. This is above the paper's own significance level of 0.05. None of the usual alternative tests for a 2 × 2 table gets below 0.01 either; the smallest, a one-sided two-proportion z-test, gives 0.015. The comparison is the basis for the paragraph's conclusion that MnVGs "significantly outperformed conventional electrospinning grafts". The counts show a large observed difference (90% vs 50%), but with six control grafts the stated test does not reach significance.

What the paper reports

Results, section "MnVGs maintained high patency rates and rapid neo-adventitia Ingrowth", first paragraph:

15 mm segmental defect was surgically created in the left carotid artery of each rabbit, with MnVGs implanted in the experimental group (n = 20) and electrospinning grafts as controls (n = 6) (Fig. 2a, b). At 2 weeks after implantation, graft performance diverged markedly between groups.

Doppler ultrasound revealed a 90% patency rate (18/20) in the MnVGs group whereas controls exhibited 50% patency (3/6) and significant regurgitation despite of administration of anticoagulating agents (p < 0.01, Fisher’s exact test).

The last sentence of the same paragraph:

Collectively, these results demonstrated that MnVGs significantly outperformed conventional electrospinning grafts, with structural openness critically enhancing patency and regenerative efficiency in 2 weeks.

Abstract:

When implanted in rabbits, MnVGs rapidly regenerated neo-adventitia within the highly porous graft walls and achieved endothelialization as early as 2 weeks postoperatively, resulting in 90% patency rate (18/20).

Discussion, fourth paragraph:

The key finding of this study is that MnVGs achieved rapid endothelialization within 2 weeks postoperatively, resulting in a 90% patency rate (18/20) in 1.5 cm carotid artery reconstruction in rabbits.

Methods, "Animal experiments":

Male New Zealand white rabbits (6 months old, weighing 3.5-4.5 kg; n = 32) underwent implantation of 15 mm long MnVGs interposition grafts in the left carotid artery. The experimental cohorts included a control group (Con-2W; n = 6) and a test group (MnVGs; n = 26).

Methods, "Statistical analysis":

Group comparisons were assessed using the Student’s t-test for two groups and one-way analysis of variance (ANOVA) followed by Tukey’s post-hoc test for multiple groups. Statistical significance was set at P < 0.05.

What the paper's own numbers show

Method. I built the 2 × 2 table from the counts in the Results sentence: MnVGs 18 patent and 2 occluded, controls 3 patent and 3 occluded. The Source Data file (sheet "Figure 2", "Figure 2e Number of graft") gives the same counts: 2 weeks, 18 patent and 2 occluded; con-2 weeks, 3 patent and 3 occluded. I computed Fisher's exact test by hand from the hypergeometric distribution and with scipy.stats.fisher_exact. I then computed every common alternative for a 2 × 2 table, the odds ratio with its exact confidence interval, and how much the counts would have to change for Fisher's p to fall below 0.01. Every number below is printed by recompute.py.txt.

With margins fixed (21 patent and 5 occluded among 26 grafts, 20 of them MnVGs), the number of patent MnVGs can only be 15 to 20. The observed 18 has probability 13,300/230,230 = 0.0578. Only 19 (0.0046) and 20 (0.0001) are as extreme or more so, so the p-value is (13,300 + 1,050 + 21)/230,230 = 14,371/230,230 = 0.0624. The opposite tail adds nothing, because every table with 15 to 17 patent MnVGs is more probable than the observed one. That is why the one-sided and two-sided p are the same.

Quantity Location Reported Recomputed from the paper's own numbers
MnVG patency Results; Abstract; Discussion 90% (18/20) 18/20 = 90.0% (consistent)
Control patency Results 50% (3/6) 3/6 = 50.0% (consistent)
Fisher's exact p, two-sided Results p < 0.01 0.0624
Fisher's exact p, one-sided (alternative convention) p < 0.01 0.0624
Fisher mid-p, one-sided (doubled for two-sided) (alternative convention) p < 0.01 0.0335 (0.0671)
Pearson chi-square, uncorrected / Yates-corrected (alternative test) p < 0.01 0.0292 / 0.1119
Two-proportion z-test (z = 2.18), two-sided / one-sided (alternative test) p < 0.01 0.0292 / 0.0146
Barnard exact, two-sided (alternative test) p < 0.01 0.0384
Boschloo exact, two-sided / one-sided (alternative test) p < 0.01 0.0593 / 0.0296
Odds ratio (sample; conditional MLE with exact 95% CI) not reported not reported 9.00; 7.97 (0.64 to 136.7)
Difference in patency (Newcombe 95% CI) not reported not reported 0.40 (0.029 to 0.721)
MnVG count needed for Fisher p < 0.01, with controls at 3/6 sensitivity n/a 20/20 (p = 0.0077); 19/20 gives 0.0278
Control count needed for Fisher p < 0.01, with MnVGs at 18/20 sensitivity n/a 1/6 or fewer (1/6: p = 0.0018); 2/6 gives 0.0129
MnVGs out of the Methods' n = 26, vs 3/6 Methods, "Animal experiments" n/a 24/26: p = 0.0342; 25/26: 0.0149; only 26/26 gives p < 0.01 (0.0040)

The stated test gives p = 0.062, about six times the printed bound and above the paper's α of 0.05. The exact 95% confidence interval of the odds ratio (0.64 to 136.7) includes 1, which agrees with that test. The smallest p-value of any variant is 0.015, from a one-sided large-sample z-test. That test is not the one reported, and its normal approximation is poor with three occluded grafts among six. Some of the less conservative methods (uncorrected chi-square, mid-p, Barnard, the Newcombe interval for the difference) put the result just below 0.05. None of them supports p < 0.01.

Figure

The printed bound p < 0.01 lies well below every p-value recomputed from 18/20 vs 3/6; the stated Fisher's exact test gives 0.062

Figure 1. P-values for the 2-week patency comparison (18 of 20 MnVGs patent vs 3 of 6 controls), on one log-scale axis. The orange arrow is the value printed in the Results, "p < 0.01, Fisher’s exact test", drawn as the interval from 0.01 downward. The blue markers are p-values recomputed from the paper's counts. The filled marker is the test the paper names, Fisher's exact test, two-sided (0.062); open markers are common alternatives. The dotted line marks 0.01 and the dashed line marks the paper's significance level of 0.05. Every recomputed value lies to the right of the printed bound. The log scale is used because p-values span orders of magnitude; it does not exaggerate the gap, which is a factor of about 6 between 0.01 and 0.062.

Fisher's p falls below 0.01 only if all 20 MnVGs had been patent

Figure 2. Fisher's exact two-sided p, recomputed for each possible number of patent MnVGs out of 20, with the controls held at the reported 3 of 6. The reported counts (18/20, filled marker) give 0.062. Only 20/20 would give p < 0.01 (0.008, orange line), and only 19/20 or more would give p < 0.05. Holding the MnVGs at 18/20 instead, the controls would need 1 of 6 patent or fewer (see table).

Why it matters

Explanations considered

Status

Suggested correction

In Results, section "MnVGs maintained high patency rates and rapid neo-adventitia Ingrowth", replace

(p < 0.01, Fisher’s exact test)

with

(p = 0.062, two-sided Fisher’s exact test)

and revise the paragraph's last sentence so that it does not describe the patency difference as significant. For example:

Collectively, these results suggested that MnVGs outperformed conventional electrospinning grafts in 2-week patency (90% vs 50%), although the difference did not reach statistical significance with six control grafts.

The authors may also wish to state the number of MnVG rabbits consistently between the Results (n = 20) and the Methods (n = 26), and to add Fisher's exact test to the Statistical analysis section, which currently lists only t-tests and ANOVA.