A neuropsychologist created a way to detect fake academic research and found a shocking amount of it

One in five scholarly journal articles may contain data created by a “paper mill” that was paid to make it up, a new study from German researchers says.

The new research adds to the increasingly alarming realization that academia is in the throes of a fake-research problem, one being exacerbated by the rise of AI. “Because the scale of fake publications in biomedicine is unknown,” the authors explain, “we developed a simple method to red-flag them and estimate their number.”

Their preprint study, released this week on the site medRxiv, analyzed 15,120 papers that were published in 2020 and searchable on PubMed, a database where you can find 35 million citations for the biomedical sciences. Their model flagged thousands of articles as potentially being made-up or plagiarized, a proportion that they call “considerable” and “on the rise.”

“It is just too hard to believe,” lead author Bernhard Sabel, head of Otto von Guericke University Magdeburg’s Institute of Medical Psychology, told the journal Science. He likened the findings to “somebody tell[ing] you 30% of what you eat is toxic.”

He and his coauthor, the University of Potsdam psychologist Gerd Gigerenzer, note that this doesn’t unequivocally establish these papers are fraudulent—that would be a Herculean task. Instead, their method accounts for genuine papers that the model mis-flagged as fake, and concludes this yields an actual rate of scholarly bullshit closer to 20%, which in this case is about 300,000 of the 1.3 million biomedical papers published every year.

Their method for determining if papers were fake began with questions that functioned like a scholarly challenge-response spam filter: Did the author use a private email address? (This should be a “no.”) Were they affiliated with an actual hospital or research institution? Did they provide their original data to their editor? Were they willing to answer the journal’s quality-check survey? (These should be “yes.”) The team also weighed where the papers appeared, assuming journals whose articles get less attention would be less competitive and more desperate for content.

There could be myriad reasons why a journal or author might stoop to publishing a fake article, but the publish-or-perish culture present at many universities and research institutions is probably the easiest to blame. These days, scientists, grad students, and other faculty are sometimes required to hit a fixed number of first-authored articles to qualify for promotion.

Meet the fraud factories

Paper mills (or “editing services,” in their own terminology) have emerged to lend a hand, the authors say: “The dramatic rise of fake science publishing is driven by an unscrupulously corrupt—and increasingly successful—paper mill industry.”

These mills often hire science graduates to write the articles, or even real researchers looking for work, and in some cases guarantee publication in a certain caliber of journal, all for a hefty fee (as much as $25,000).

Specific services can even include making data via fabricated “experiments.” In the study, the authors publish an email that one of them, who edits a biomedical journal, received from a Guangzhou, China-based paper mill. The mill representative noted that the company cranks out over 100 manuscripts per month, and employs 100 full-time staff plus 300 freelancers in China, the United States, Canada, Turkey, India, and other countries whose goal is to help clients “get maximum exposure.”

The authors write that they’re aware of additional cases in which a paper mill attempted to bribe journal editors, usually by explaining the mill is affiliated with “friendly” journals that will cite the fake paper, thereby boosting its citation impact.

The study also tracked down where these paper mills are located. Companies in China were responsible for 56% of the fake articles that the authors identified. Next came mills in the United States (responsible for 7.3%), India (6.8%), and Europe (6.6%).

Underscoring what a lucrative cottage industry this scam has morphed into, the study adds that between 40% and 50% of the entire biomedical-journal-article output of countries like Russia, Turkey, Egypt, and China appears to be fake. (The U.S.’s ratio in the study was 3.2%.)

To most laypeople, the findings are less of a warning about reading Nature, Cell, or other established journals than about casually googling topics, then wading through unfiltered results. The authors conclude by noting the “collateral damage” posed by fake science—more experiments and clinical trials may fail, public health information might be less accurate, therapies presumed safe and effective maybe won’t be, and the field “runs the risk that the public loses its trust in the honesty of science itself.”

Some journals seem to have started to appreciate the problem: Last week, Hindawi, a large publisher of hundreds of scholarly journals that’s been accused of predatory publishing, agreed to close down four journals that it admitted were “heavily compromised by paper mills.”