An AI audit of published research
We scanned 438,528 scientific papers. 4,482 contradict their own data.
Each one is a place where a paper’s own numbers disagree with what it says: a summary that its tables don’t support, a calculation that doesn’t add up, a result called clear when its own figures say it could be chance. Two AI judges, working separately, rated every one a major error. Here are some of the clearest.
- 438,528 research papers from 129 leading journals, each read in full by an AI model
- 89,462 had something that looked wrong
- 10,942 still looked wrong when two AI models checked again, thinking harder, each on its own
- 7,347 the most promising, reviewed so far by two stronger AI judges
- 4,482 both judges agree: a real, major error
- 252 checked in depth so far, every number re-derived from the paper
Each square’s area is the number of papers at that step. Papers come from the open-access collection of PubMed Central.
What we found
“The summary” on these cards is the paper’s abstract: the short summary printed at its top, which is the part most readers see.
Building energy · Nat Commun · 2025
Claimed smart-window saving: 34 times what US commercial buildings use. A million printed as a billion.
Energy per year in megawatt-hours (MWh); squares drawn to scale by area
Cancer genetics · JAMA · 2010
Paper: chance rarely makes its kidney-cancer gap between gene groups. Its own counts: 1 in 4.
How often chance alone makes a 9% vs 4% gap when risks are equal
Child health · JAMA Netw Open · 2020
“69% less likely to be hospitalized.” Its own figure, 0.59 times as likely, means 41% less.
Chance of a hospital stay within a year, compared with the other group’s
Vaccine research · Nat Commun · 2023
Vaccine paper: chance gives its result under 1 in 10,000. Its own test: 1 in 8.
How often chance alone would produce this gap, if the vaccine did nothing
Breast cancer · BMC Med · 2026
The summary reports no severe hand-foot skin reactions. The paper’s own table: 5 of 26 patients.
Share of patients with hand-foot syndrome; severe means grade 3, the top level
Carbon storage · Sci Adv · 2024
Paper: rock stores 15 billion tonnes of COâ‚‚ per cubic kilometre. Its own numbers: 16 million.
Tonnes in one cubic kilometre of rock; squares drawn to scale by area
Cancer care · Lancet Reg Health Eur · 2025
The paper says 707 of 1,109 patients died within 90 days. Its own counts give 408.
Patients who died within 90 days, of all 1,109
Public health · Lancet Reg Health Am · 2025
Vaccines “prevented 144%” of intensive care admissions, which is impossible. The paper’s own counts give 59%.
Intensive care admissions vaccines prevented, as a share of the 22,017 expected without them
Vaccines · Cell · 2020
BCG “can protect the elderly against infections”? By the paper’s own rule, it could be luck.
How fast the vaccinated caught infections, as a multiple of the dummy group’s rate
Coastal science · Nat Commun · 2023
Summary: “the majority” of muddy coasts erode over a metre a year. Its own table: 30.5%.
Share of all the world’s muddy coastline, 1984–2016; the three groups add to 100%
Lung medicine · Eur Respir J · 2022
Summary: fewer hospital stays and deaths in 2020, not luck. Its own counts: could be luck.
How often chance alone would give a gap this big in heart-failure hospital stays
Health spending · Lancet Reg Health Am · 2024
The paper’s summary says Brazil’s lung-TB costs “exceeded $1.3 billion.” Its own results say $108 million.
Total spent on lung TB, 2015–2022, US dollars; each square’s area matches its amount