Vaccine research · Nat Commun · 2023
Vaccine paper: chance gives its result under 1 in 10,000. Its own test: 1 in 8.
How often chance alone would produce this gap, if the vaccine did nothing
The paper says
After nine exposures to SIV, a monkey version of HIV, 3 of 7 vaccinated monkeys stayed uninfected; 0 of 6 unvaccinated did. The paper says chance alone would produce a gap this big under 1 time in 10,000, and its title says the vaccine “protects”.
Its own data show
The paper names its test, Fisher’s exact test. On its own counts it asks: if the vaccine did nothing, how often would all 3 uninfected monkeys be vaccinated ones? About 1 time in 8 (35 in 286). Even all 7 vaccinated staying uninfected would give 1 in 1,716: this test cannot reach 1 in 10,000.
A p-value is how often chance alone would produce a gap this big if the vaccine did nothing. The paper’s own bar is under 1 in 20. About 1 in 8 misses it: the gap could easily be chance. 4 of 7 vaccinated uninfected would have passed. This is the paper’s main protection result, behind its title’s “protects”.