Paperonium

Vaccines · Cell · 2020

BCG “can protect the elderly against infections”? By the paper’s own rule, it could be luck.

Its table, at theeveryday 1-in-20 test“significant”Worked out at its 1-in-185 testcould be no difference0.31 to 0.97×0.24 to 1.24×0.2×0.3×0.5×0.7×1×1.5×vaccine made no difference
Its table, at theeveryday1-in-20 test“significant”Worked out atits 1-in-185 testcould be nodifference0.97×0.310.24 to 1.24×0.2×0.5×1×2×vaccine madeno difference

How fast the vaccinated caught infections, as a multiple of the dummy group’s rate

The paper says

Elderly hospital patients got BCG, a tuberculosis vaccine, or a dummy shot. At a mid-trial check, the vaccinated were catching infections of any kind about half as fast (0.55 times the dummy group’s rate). The paper calls this “significant”, meaning unlikely to be luck.

Its own data show

Its table gives the chance that luck alone, if BCG did nothing, would produce a gap this big, favouring either group: about 1 in 26. That is rare enough for the everyday 1-in-20 test, but not the far stricter 1-in-185 test the paper’s own plan sets for a mid-trial check. It never mentions missing it.

Every look at the data gives luck another chance to look like a real effect, like rolling twice for a six, so mid-trial checks need much stronger evidence. BCG may still work; this check can’t tell a real benefit from luck. Yet the summary says BCG “can protect the elderly”, and no final results have been posted.

“The summary” means the paper’s abstract: the short summary printed at its top, which is the part most readers see.

Read the paper →Full technical write-up →

Activate: Randomized Clinical Trial of BCG Vaccination against Infection in the Elderly · Cell (2020) · doi:10.1016/j.cell.2020.08.051