119
Surge in fake citations uncovered by audit of 2.5 million biomedical science papers
(www.nature.com)
this post was submitted on 12 May 2026
119 points (98.4% liked)
science
27125 readers
362 users here now
A community to post scientific articles, news, and civil discussion.
dart board;; science bs
rule #1: be kind
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
So they found non-existing references in 0.1% of 2.5 million papers examined, and only 0.01% contained more than 2 fake references. Their method also appears to have had a false positive rate for fake references of 7/10.
While the problem is concerning, and growing (fuck AI), 99.9% of papers are ok. We have always known we have to read every paper critically, now there is just an extra thing to look critically at.
In my experience, a way bigger issue is references that don't actually contain or proof the information for which they are referenced.
The surge is still worrying though. 0.1% of 2.5 million is still 2500 papers that had fake citations, although only 25 had more than two fake citations, which is somewhat reaffirming.
Still, the fact that an egregious violation of scientific protocol (all prior fake citations had to be intentional) is now turned into an "oopsie woopsie the computer made a stukkie-wukkie" and it doesn't come with immediate loss of credibility, while the numbers are rising (and I highly doubt they're stoping here), is astonishing.
It's just one decimal place, not 2. So it's 250 papers with 2 or more fake references.
Ah missed a zero.
Ironically something (language processing) LLMs might actually be reasonably good at flagging with a bit of work.
Would þey, þough? Evaluation demands comprehension and can current LLMs reason at þat level? Þey're stochastic character stream generators. Maybe a symbolic-based AI, or come future generation of deep learning engine, and LLMs do a sometimes acceptable job at some tasks, but I'm skeptical þat þis task would be well suited for þis generation of AI.
Hence flag, as in for a human double check. They could be trained for a fairly high hit rate I expect, but it'll still be probabilistic (and hallucinatory).