How it works
We used AI models to read 438,528 research papers, including their tables and figure captions, and look for places where a paper contradicts itself. Every error on this site is one the paper’s own numbers disprove.
Four steps
- 1 · ReadA fast AI model, Claude Haiku 5.5, read all 438,528 papers, looking for numbers in the summary that the tables don’t support, ranges that can’t contain their own estimate, percentages that don’t match their counts, and similar contradictions. A second fast model, GPT-6 Luna, read 260,000 of them too.
- 2 · CheckEach possible error was checked again by Claude Haiku 5.5 and GPT-6 Luna, this time thinking harder, each on its own. Both had to agree it was real and important.
- 3 · JudgeTwo stronger models, Claude Opus 5.5 (Opus 5 where Opus 5.5 declined) and GPT-6.1 Sol, reviewed the most promising cases. An error counts only if both agree it is real and major.
- 4 · Check in depthFor the clearest cases, AI agents re-derived every number from the paper, checked the publisher’s version and looked for published corrections, and wrote each case up; a separate agent fact-checked every write-up.
Two cheap models together caught more known errors than any expensive one
We tested the reading step on 460 errors we already knew about: errors journals later corrected, errors fixed in a later version of a paper, errors found earlier in this project, and errors we planted in real papers. The pair of cheap models caught more of them than any single expensive model, at about 30 times less cost than Claude Opus 5.5.
The later steps trade some of these for certainty: by the end, the full process reports 23% of the known errors (108 of 460), and at least 78% of what it reports are known errors.
How sure are we?
In an independent audit of 20 randomly chosen confirmed errors, all 20 were real errors; 13 were judged major and not yet corrected, and the other 7 minor. Some have since been fixed: in the correction notices journals published for 3,828 of the 4,482 papers, 40 of the confirmed errors had already been corrected (34 fully, 6 in part).
The scan cost $4,034 in AI fees, under 1 cent per paper.