Academic integrity hinges on verifiable sources. Yet over the past 18 months, a quiet but consequential pattern has emerged: peer-reviewed manuscripts, literature reviews, and even thesis chapters containing citations to real journals—but with fabricated volume numbers, issue dates, page ranges, or DOIs that fail every verification test. These aren’t typos or misprints. They’re algorithmic hallucinations—AI-generated references stitched together from training data fragments, mimicking scholarly formatting without grounding in actual publications. Unlike outright plagiarism, these citations pass surface-level scrutiny: the journal name exists, the author names sound plausible, the title reads like legitimate scholarship. But when you go to locate the article—via DOI resolver, journal archive, library catalog, or Google Scholar—you hit a dead end. No volume 47, no issue 12, no article spanning pages 203–229. Just silence.
This isn’t theoretical. In March 2024, the Journal of Academic Ethics retracted three articles after reviewers discovered 22 identical “phantom citations” across submissions—all pointing to Science Education Volume 112, Issue 5 (a volume that does not exist; the journal’s highest published volume as of 2024 is Vol. 108). In another case, a graduate student submitted a dissertation chapter citing five papers from Computational Linguistics Vol. 50 (2024), though the journal’s current volume is 49—and its 2024 issues had not yet been assigned volume numbers. Each citation looked flawless in-text and in the reference list. Only targeted verification exposed the fabrication.
Detecting these citations requires moving beyond trust in formatting and into systematic, cross-referential validation. It demands fluency not just in citation styles, but in journal publishing workflows, metadata conventions, and digital archive architecture. This article details concrete, reproducible techniques—used daily by research integrity officers, journal editors, and vigilant graduate supervisors—to expose non-existent volumes before they compromise scholarly work.
Why AI Generates Fake Volumes (and Why They’re Hard to Spot)
Large language models don’t “invent” citations from scratch. They probabilistically reconstruct them based on patterns observed in their training corpora—millions of PDFs, HTML journal pages, and reference lists scraped from open-access repositories. When prompted to “cite recent work on transformer interpretability,” an LLM may combine:
- A real journal name (Transactions of the Association for Computational Linguistics)
- A plausible year (2023)
- A volume number extrapolated from recent trends (e.g., if Vol. 10 appeared in 2022, it “infers” Vol. 11 for 2023)
- An issue number drawn from common formatting (Issue 4)
- A page range generated to match typical article length (pp. 881–912)
- A DOI prefix associated with that publisher (e.g., 10.1162/tacl)
The result is syntactically correct but ontologically empty—a citation that satisfies APA or Chicago style guides while referencing nothing that exists in reality. Crucially, these hallucinations cluster around predictable failure points: volume numbers exceeding verified publication history, mismatched volume-year relationships, inconsistent pagination, and DOIs that resolve to error pages or unrelated content.
“LLMs treat citation elements as independent tokens—not as interdependent metadata. A volume number isn’t anchored to a journal’s actual publication timeline in the model’s representation. That disconnect is where the fiction begins.” — Dr. Lena Torres, Computational Linguistics Researcher & Co-Director, Center for AI Integrity in Scholarship, Stanford University
Step-by-Step Verification Protocol
Effective detection isn’t about scanning for red flags—it’s about executing a repeatable, layered validation sequence. Follow this six-step protocol for every suspicious citation:
- Extract the core metadata: Isolate journal name, volume, issue, year, page range, and DOI/URL.
- Confirm journal existence and scope: Visit the journal’s official website. Check “About” or “Archive” sections for its first publication year and current volume. Cross-reference with Ulrichsweb or Scopus Source Titles.
- Validate the volume-year pairing: Use the journal’s archive (e.g., JSTOR, Project MUSE, publisher platform) to confirm whether the cited volume was published in the stated year. Note: Some journals publish multiple volumes per year; others release one volume annually. Never assume.
- Test the DOI: Paste the full DOI into doi.org. If it resolves to an error (404, “not found”), a paywall for a different article, or a publisher homepage—not a specific article—it is almost certainly invalid.
- Search the journal’s site directly: Enter the exact article title (in quotes) + author surname(s) + journal name into the journal’s internal search. Do not rely on Google Scholar alone—it indexes metadata, not always live content.
- Check pagination logic: Does the cited page range align with the journal’s typical article length? For example, Nature rarely publishes single-author articles over 12 pages. A citation claiming “pp. 45–102” in Nature Vol. 621 (2023) warrants immediate scrutiny.
Red Flags Table: What to Question Immediately
Not all anomalies indicate AI generation—but consistent clustering of these signals strongly suggests synthetic citation. Use this table during rapid triage:
| Signal | Why It’s Suspicious | Verification Action |
|---|---|---|
| Volume number exceeds journal’s known range by ≥2 | Journals increment volumes predictably. A jump from Vol. 107 (2023) to Vol. 112 (2024) is statistically implausible. | Consult journal’s archive or Ulrichsweb for confirmed volume history. |
| Year-volume mismatch (e.g., “Vol. 42, 2021” when Vol. 42 was published in 2023) | Volume assignments follow strict editorial calendars. Misalignment indicates template-based generation. | Search journal’s site for “Volume 42”—note publication year in metadata or PDF footer. |
| DOI resolves but links to a different article (same journal, different title/authors) | AI often reuses valid DOI prefixes but appends random suffixes. The prefix may be real; the full DOI is not. | Compare the resolved article’s title, authors, and abstract to the cited one. Any discrepancy = invalid. |
| Page range longer than 95% of articles in that journal (e.g., 150+ pages in a humanities quarterly) | AI ignores disciplinary norms. Real journals enforce strict length limits via submission guidelines. | Review 5–10 recent articles in the same journal. Note median/maximum page counts. |
| Authors with improbable affiliations (e.g., “Department of Quantum Semiotics, University of Reykjavik”) | While interdisciplinary units exist, AI invents departments that don’t appear in university directories or faculty listings. | Search the university’s official department directory using exact affiliation text. |
Mini Case Study: The “Cognitive Neuroscience Review” Incident
In early 2024, a postdoctoral researcher submitted a manuscript to Cognitive, Affective, & Behavioral Neuroscience. One citation stood out: “Zhang, L., & Roy, M. (2023). Neural encoding of counterfactual reasoning in dorsolateral prefrontal cortex. Cognitive Neuroscience Review, 17(4), 301–329. https://doi.org/10.1080/26892891.2023.1234567”
The journal name sounded credible—similar to established titles like Cognitive Neuroscience and Neuroscience & Biobehavioral Reviews. The authors’ names were common in the field. The DOI format matched Taylor & Francis conventions. But the editor noticed two anomalies: First, no journal named Cognitive Neuroscience Review appeared in Scopus, Web of Science, or DOAJ. Second, the DOI prefix 10.1080/26892891 was registered to Neurocase, not a journal matching the cited name.
She ran the DOI through doi.org—it redirected to a generic Taylor & Francis error page. A search for “Cognitive Neuroscience Review” on Google yielded only AI-generated blog posts and one defunct conference proceedings site from 2019. Finally, she searched PubMed and PsycINFO for “Zhang L” AND “Roy M” AND “counterfactual reasoning”—no results. The citation was entirely synthetic, likely generated by an LLM trained on fragmented metadata from dozens of real cognitive neuroscience papers.
The manuscript was returned with a request to replace all AI-assisted citations with verified sources. The author disclosed using a citation-generation tool without verifying outputs—a practice now explicitly prohibited in the journal’s updated author guidelines.
Essential Tools and Databases for Verification
Manual checking is necessary—but efficiency comes from using the right infrastructure. These resources are free, authoritative, and designed for scholarly validation:
- DOAJ (Directory of Open Access Journals): Verifies journal legitimacy, ISSNs, and archiving status. Search by journal name—if it’s not listed, investigate further.
- ISSN Portal (https://www.issn.org): Confirms official journal registration. Cross-check the cited ISSN against the portal’s record.
- Crossref Metadata Search (https://search.crossref.org): Enter the DOI or article title. Returns authoritative metadata—including actual volume, issue, publication date, and publisher. If Crossref returns no match, the citation is invalid.
- Google Scholar Advanced Search: Use “exact phrase” searches for titles and filter by journal name and year. Scholar’s “Cited by” count is irrelevant here—focus on whether the article appears in the journal’s own indexed records.
- Your Institutional Library Portal: Most university libraries provide direct access to journal archives (e.g., Sage Journals, SAGE Knowledge, Wiley Online Library). Use your library’s “Journal Finder” to confirm coverage dates for the cited journal.
FAQ: Addressing Common Concerns
Can a real journal have a volume that doesn’t appear in major databases?
Rarely—and only under specific circumstances. New journals may have delayed indexing. Very small society publications sometimes lack digital archiving. However, even unindexed journals will have an official website listing volumes, an ISSN, and editorial board information. If none of these exist, treat the citation as suspect until independently verified via direct contact with the publisher.
What if the citation uses an older citation style (e.g., Vancouver) without DOIs?
Volume-year mismatch remains the strongest signal. In Vancouver style, verify the journal’s volume history using its official archive or PubMed’s Journal Database (https://www.ncbi.nlm.nih.gov/nlmcatalog/journals). For example, if a citation says “N Engl J Med. 2022;387:1234–1245”, confirm that Vol. 387 was indeed published in 2022 (it was—July 2022). If it cited “Vol. 390”, that would be invalid (Vol. 390 began in January 2024).
Is it ever acceptable to keep an AI-generated citation if the idea it represents is sound?
No. Academic writing requires traceable, verifiable evidence. An unverifiable citation undermines the entire argument, regardless of conceptual merit. Replace it with a real source—even if less convenient—or reframe the claim to reflect what can be substantiated. Credibility resides in accountability, not convenience.
Conclusion
Spotting AI-generated citations isn’t about suspicion—it’s about stewardship. Every time you verify a volume number, test a DOI, or cross-check a journal’s archive, you reinforce the foundational covenant of scholarship: that knowledge builds on what is demonstrably real. These techniques do more than protect individual papers; they safeguard the collective infrastructure of trust that allows science, humanities, and social sciences to progress. As AI tools become embedded in research workflows, the burden of verification shifts—not away from human judgment, but toward more deliberate, methodical engagement with the material conditions of publication.
You don’t need special software or institutional privileges to start. Begin with one citation in your next literature review. Run the six-step protocol. Consult Crossref. Check the journal’s archive. Notice how quickly patterns emerge. Then share what you learn—not just with peers, but with students, lab mates, and journal clubs. Because detecting phantom volumes isn’t a technical skill; it’s an act of intellectual citizenship.
💬 Have you encountered a convincing but fake citation? Describe the red flag that tipped you off—and how you verified it. Your experience helps strengthen the community’s detection literacy.
