‘Tip of the Iceberg’: Study Uncovers AI-Fabricated Citations in Research Papers

Fabricated citations are an increasing problem coinciding with the widespread use of artificial intelligence (AI) tools in writing and editing, an analysis of biomedical research papers showed.

Out of 97.1 million verified references on biomedical papers, 4,046 fabricated references across 2,810 papers were identified, reported Maxim Topaz, PhD, MA, RN, of Columbia University in New York City, and colleagues.

About one in 2,828 papers contained at least one fabricated reference in 2023, and by 2025, this had increased to one in 458, and to one in 277 papers in the first 7 weeks of 2026, they wrote in correspondence to The Lancet.

The fabrication rate increased by more than 12-fold, from 4.0 in 10,000 papers in 2023 to 56.9 per 10,000 papers in early 2026.

“I think we are catching just the tip of the iceberg,” Topaz told MedPage Today.

Fabricated references have titles that don’t correspond with an existing publication. Large language models (LLMs) are known for hallucinating plausible-sounding references that “are often correctly formatted, attributed to real researchers, and bear plausible publication dates, making them difficult to detect by conventional peer review,” the authors explained, noting that the spike in fabricated references in mid-2024 coincided with the publication lag following growing LLM adoption.

Topaz told MedPage Today that this research stemmed from his own experience. He regularly uses AI tools to polish his writing. After submitting a paper to a journal, the editorial staff alerted him that one of his citations didn’t lead to a source. Topaz said this “deeply embarrassed” him and inspired him to look further.

He and his team developed an automated reference verification system and scanned PubMed Central’s Open Access subset. They looked at nearly 2.5 million biomedical papers published from January 2023 through February 2026, which had 125.6 million references; 77% had a PubMed identifier (PMID) and were verified; the rest were excluded. For each of the verified references, the team compared “the citing paper’s claimed metadata with the use of text-similarity scoring” and flagged mismatches.

To minimize false positives based on formatting errors, flagged references underwent more filters. References that passed all filters were verified against PubMed, Crossref, OpenAlex, and Google Scholar. References that were not found in any database were considered fabricated and those found but linked to an incorrect identifier were considered reference errors.

In one particularly alarming example, 60% of the references on a 2025 paper on ureteroileal anastomotic techniques in an open access oncology journal were found to be fabricated. Each of the references was tailored to the paper’s narrow surgical topic and attributed to real urologists.

The researchers also found patterns consistent with paper mill activity, like an author duo appearing across 11 papers in a single surgical journal in 2025 with 15 fabricated references.

Topaz said their findings emphasize that humans need to double check all parts of the research process “because AI can hallucinate at any point, and we need to really be careful about that.” He said this responsibility falls to both authors and journals.

In an accompanying editorial, Howard Bauchner, MD, of Boston University Chobanian and Avedisian School of Medicine, and Frederick Rivara, MD, MPH, of the University of Washington and Seattle Children’s Research Institute, argued that the onus for catching fabricated citations falls on authors.

They noted that fabricated references are only going to get worse with the increasing use of AI and LLMs in writing and editing.

Bauchner and Rivara, both former JAMA editors, stressed that papers in which a “fabricated reference is detected should be retracted.”

“A fabricated reference in a published manuscript is consistent with research misconduct and should be treated as such,” they wrote.

Arjun K. Manrai, PhD, of Harvard Medical School in Boston, who was not involved with the research, told MedPage Today that many AI researchers have known that fabricated references are a problem, yet it’s barely been studied.

“Fabricated references should concern us all,” Manrai said. “If our scientific foundation erodes, we are in deep trouble.”

Topaz and co-authors recommended four strategies for handling the problem of fabricated citations. First, publishers should have automated reference verification before peer review. Second, indexing services should add integrity metadata to article records. Third, publishers should retroactively screen already published research. And finally, fabricated references should be established as a category in research integrity databases.

This study is limited by the exclusion of nearly a quarter of references that didn’t have a PMID, which could bias estimates. Plus, the researchers only looked at PubMed Central, which is just a subset of all biomedical research.

Topaz said the team is doing further analysis and finding that fabricated citations are proliferating further down in the literature into systematic reviews and clinical guidelines.

“This is the most alarming piece, where you would find yourself providing clinical care as a clinician based on the guideline, and then you look at the … articles that are cited in this guideline … and you see that those citations lead to nowhere,” Topaz said. “That’s very concerning.”

Please enable JavaScript to view the comments powered by Disqus.