Paperonium

An AI audit of published research

We scanned 438,528 scientific papers. 4,482 contradict their own data.

Each one is a place where a paper’s own numbers disagree with what it says: a summary that its tables don’t support, a calculation that doesn’t add up, a result called clear when its own figures say it could be chance. Two AI judges, working separately, rated every one a major error. Here are some of the clearest.

How the scan narrowed 438,528 papers down to 4,482
  1. 438,528 research papers from 129 leading journals, each read in full by an AI model
  2. 89,462 had something that looked wrong
  3. 10,942 still looked wrong when two AI models checked again, thinking harder, each on its own
  4. 7,347 the most promising, reviewed so far by two stronger AI judges
  5. 4,482 both judges agree: a real, major error
  6. 252 checked in depth so far, every number re-derived from the paper

Each square’s area is the number of papers at that step. Papers come from the open-access collection of PubMed Central.

What we found

“The summary” on these cards is the paper’s abstract: the short summary printed at its top, which is the part most readers see.

Building energy · Nat Commun · 2025

Claimed smart-window saving: 34 times what US commercial buildings use. A million printed as a billion.

The summary: claimed saving66.9 billion MWh34 times what US commercialbuildings useWhat US commercial buildings useabout 2.0 billion MWhIts own numbers66.9 million MWhenlarged 4× in width
The summary: claimed saving66.9 billion MWh34 times what US commercial buildings useWhat US commercialbuildings useabout 2.0 billion MWhIts own numbers66.9 million MWhenlarged 4× in width

Energy per year in megawatt-hours (MWh); squares drawn to scale by area

What went wrong →

Cancer genetics · JAMA · 2010

Paper: chance rarely makes its kidney-cancer gap between gene groups. Its own counts: 1 in 4.

1 in 1001 in 1,000every time1 in 20the usual barabout1 in 250The paper’s text: passesabout1 in 4Its own counts: fails
1 in 1001 in 1,000every time1 in 20the usual barabout1 in 250The paper’s text:passesabout1 in 4Its own counts: fails

How often chance alone makes a 9% vs 4% gap when risks are equal

What went wrong →

Child health · JAMA Netw Open · 2020

“69% less likely to be hospitalized.” Its own figure, 0.59 times as likely, means 41% less.

The summary: “69%less likely”would need 0.31×Its own figure: helped children41% lessSwapped: no-helper vshelped children1.69× = 69% more0.31×0.59×0.380.940.3×0.5×0.7×1×1.5×2×same chanceless likelymore likely
The summary:“69% less likely”would need 0.31×Its own figure:helped children41% lessSwapped:no-helper vshelped children1.69× = 69% more0.31×0.59×0.380.940.2×0.5×1×2×same chanceless likelymore likely

Chance of a hospital stay within a year, compared with the other group’s

What went wrong →

Vaccine research · Nat Commun · 2023

Vaccine paper: chance gives its result under 1 in 10,000. Its own test: 1 in 8.

1 in 1001 in 1,0001 in 10,000every time1 in 20the usual barrarer than1 in 10,000The paper’s textabout1 in 8Its own test, on its 13 monkeys
1 in 1001 in 1,0001 in 10,000every time1 in 20the usual barrarer than1 in 10,000The paper’s textabout1 in 8Its own test,on its 13 monkeys

How often chance alone would produce this gap, if the vaccine did nothing

What went wrong →

Breast cancer · BMC Med · 2026

The summary reports no severe hand-foot skin reactions. The paper’s own table: 5 of 26 patients.

Severe (grade 3)noneThe summary5 of 26Its own table: 19.2%too lowSchedule picked for next phase: severe27.3%3 of 11Any severity50%Its own table: 13 of 260%204060
Severe (grade 3)noneThe summary5 of 26Its own table: 19.2%too lowSchedule picked for next phase: severe27.3%3 of 11Any severity50%Its own table: 13 of 260%204060

Share of patients with hand-foot syndrome; severe means grade 3, the top level

What went wrong →

Carbon storage · Sci Adv · 2024

Paper: rock stores 15 billion tonnes of COâ‚‚ per cubic kilometre. Its own numbers: 16 million.

The paper’s text15 billion tonnes of CO2five times what the rock weighsWhat the rock weighs3 billion tonnesIts own numbers16 million tonnesenlarged 4× in width
The paper’s text15 billion tonnes of CO2five times what the rock weighsWhat the rock weighs3 billion tonnesIts own numbers16 million tonnesenlarged 4× in width

Tonnes in one cubic kilometre of rock; squares drawn to scale by area

What went wrong →

Cancer care · Lancet Reg Health Eur · 2025

The paper says 707 of 1,109 patients died within 90 days. Its own counts give 408.

707The summary and results299deaths too many408Its own risk groups: deaths, added up362high-risk39middle-risk7low-risk
707The summary and results299deathstoo many408Its own riskgroups: deaths,added up362high-risk39middle-risk7low-risk

Patients who died within 90 days, of all 1,109

What went wrong →

Public health · Lancet Reg Health Am · 2025

Vaccines “prevented 144%” of intensive care admissions, which is impossible. The paper’s own counts give 59%.

Share prevented59%Its own counts12,997 of 22,017The summary and results144%impossibletoo high50%75100all 22,017 expected without vaccines
Share prevented59%Its own counts12,997 of 22,017The summary and results144%impossibletoo high50%75100all 22,017 expected without vaccines

Intensive care admissions vaccines prevented, as a share of the 22,017 expected without them

What went wrong →

Vaccines · Cell · 2020

BCG “can protect the elderly against infections”? By the paper’s own rule, it could be luck.

Its table, at theeveryday 1-in-20 test“significant”Worked out at its 1-in-185 testcould be no difference0.31 to 0.97×0.24 to 1.24×0.2×0.3×0.5×0.7×1×1.5×vaccine made no difference
Its table, at theeveryday1-in-20 test“significant”Worked out atits 1-in-185 testcould be nodifference0.97×0.310.24 to 1.24×0.2×0.5×1×2×vaccine madeno difference

How fast the vaccinated caught infections, as a multiple of the dummy group’s rate

What went wrong →

Coastal science · Nat Commun · 2023

Summary: “the majority” of muddy coasts erode over a metre a year. Its own table: 30.5%.

Eroding30.5%Its own table“the majority”The summary: more than halfBarely moving40.3%Growing29.2%0%2040506080100
Eroding30.5%Its own table“the majority”The summary: more than halfBarely moving40.3%Growing29.2%0%20406080100

Share of all the world’s muddy coastline, 1984–2016; the three groups add to 100%

What went wrong →

Lung medicine · Eur Respir J · 2022

Summary: fewer hospital stays and deaths in 2020, not luck. Its own counts: could be luck.

1 in 1001 in 1,000every time1 in 20the usual barrarer than1 in 1,000The summary: fewer stays in 2020about1 in 9Its own counts: 23 vs 36 stays
1 in 1001 in 1,000every time1 in 20the usual barrarer than1 in 1,000The summary:fewer stays in 2020about1 in 9Its own counts:23 vs 36 stays

How often chance alone would give a gap this big in heart-failure hospital stays

What went wrong →

Health spending · Lancet Reg Health Am · 2024

The paper’s summary says Brazil’s lung-TB costs “exceeded $1.3 billion.” Its own results say $108 million.

The paper’s summaryover $1.3 billionHighest possible: 8 × the costliest year$171 million (a ceiling)The paper’s own results section$108 million, one-twelfth as much
The paper’s summaryover $1.3 billionHighest possible:8 × the costliest year$171 million (a ceiling)The paper’s ownresults section$108 million,one-twelfth as much

Total spent on lung TB, 2015–2022, US dollars; each square’s area matches its amount

What went wrong →
All 265 errors checked in depth →Search all 4,482 papers →