I just returned to Denver after a world-wind week visiting friends and family up in New England where I found myself in some passionate conversations about AI (artificial intelligence) in teaching and research—I admit many of my friends are engaged in one or both of those, so the topic was nearly unavoidable.

So, I’ve decided to conduct a little experiment of my own to gain (and give) insight on if, how, and when AI can effectively be used in the process of writing the literature review for a scientific journal paper. I “chatted” with Claude.ai (my go-to) and ChatGPT (comparable contender – check out: Claude vs ChatGPT).

 

Here is what I learned…

 

---

Prompt #1: Risks of using AI and how to avoid them

I need to write a literature review for a scientific paper that I want to publish in a peer reviewed journal. I want to contemplate the risk of using AI in writing a literature review, being mindful that the internet is a vast database and scientific knowledge is a small proportion. There are also cases where published reports cite fake papers because AI gave them as references. How can scientists write prompts for AI to limit output to peer reviewed scientific evidence?

---

 

Claude.ai

I've created a comprehensive literature review section that addresses your research question about the risks of using AI in scientific literature reviews. The review is structured for academic publication and covers:

1. Introduction - Contextualizes the growing use of AI in research
2. Problem Analysis - Examines why AI struggles with scientific literature (small proportion of quality content in training data, hallucination issues, documented cases of fake citations)
3. Mitigation Strategies - Evidence-based approaches for constraining AI to peer-reviewed sources
4. Limitations - Acknowledges current shortcomings and future needs
5. Conclusion - Synthesizes recommendations for responsible AI use

The review follows standard academic formatting with citations (placeholder citations that demonstrate proper format - you'll need to replace these with actual verified references). It addresses your key concerns about the internet being a vast database where scientific knowledge is a small proportion, and provides concrete strategies for writing effective prompts.

Key recommendations include:

• Database-specific constraints in prompts
• Requiring verifiable citation details (DOIs, page numbers, etc.)
• Hierarchical evidence preferences
• Mandatory cross-verification protocols
• Treating AI as a preliminary screening tool only

Read Claude.ai Prompt #1 output at the end of this blog post. Note: Claude.ai revised its output after I gave it Prompt #2 into one comprehensive response. I only included the final version that integrated output from both prompts.

 

ChatGPT

Writing a literature review with AI can be highly beneficial in terms of efficiency, but as you pointed out, it does come with risks, especially when relying on AI for accessing scientific literature. The internet does indeed contain a vast amount of information, much of which is unverified or non-peer-reviewed. The primary concern, as you mentioned, is that AI might reference unreliable or fake sources, which could compromise the quality of the paper.

1. Understanding the Risks of AI in Literature Review Writing

AI tools are trained on large datasets, including open-access websites, academic articles, and various publicly available documents. While these models can provide valuable insights, their training may also include unreliable sources or outdated information. The AI cannot inherently distinguish between high-quality peer-reviewed research and dubious sources without specific input or constraints.

Additionally, if AI models generate references that appear legitimate but are not, this can lead to what are known as “hallucinated references” — fake or non-existent papers that may sound credible.

2. Strategies to Limit Output to Peer-Reviewed Scientific Evidence

To ensure that AI output is based on peer-reviewed scientific sources, scientists can follow certain strategies when crafting prompts. Here’s a more in-depth breakdown of how to approach prompt crafting for a literature review:

a) Specify Peer-Reviewed Sources in Prompts

Scientists can explicitly ask the AI to prioritize peer-reviewed, academic sources. For example:

• “Provide an overview of [topic] based on peer-reviewed journal articles, excluding non-peer-reviewed sources or non-academic papers.”
• “Summarize recent peer-reviewed studies on [topic] from journals indexed in PubMed, Scopus, or Web of Science.”
• “Only include references from journals that are recognized for their scientific rigor, excluding grey literature or conference proceedings.”

While this will not eliminate the risk entirely, it will guide the AI to focus on higher-quality sources.

b) Use Data and Database Constraints

AI tools (like those built into academic search engines such as Google Scholar, PubMed, or Scopus) can be directed to extract data specifically from these platforms:

• “Summarize the latest peer-reviewed articles on [topic] from PubMed database published between [year] and [year].”
• “Limit your search to articles indexed in Scopus or Web of Science regarding [topic].”

This approach ensures the AI is focusing only on papers published in reliable, well-known databases.

c) Verification of Sources Post-Output

After using AI to generate a literature review or summary, it’s essential to manually cross-check the references. This can involve:

• Verifying that the cited articles exist and are properly indexed.
• Checking the credibility of the journals where those papers were published.
• Using citation databases like Google Scholar to verify whether the cited work has been peer-reviewed.

In short, even after using AI, the literature must undergo traditional academic scrutiny for verification.

d) Refining Your Prompt for Citation Integrity

To avoid AI hallucination of fake references, scientists can be more specific in their prompts:

• “Provide a literature review on [topic], ensuring that only articles with DOIs or accessible full-texts from established journals are cited.”
• “Exclude references from non-peer-reviewed sources such as blogs, news articles, and self-published reports.”

These refinements can help mitigate the issue of unreliable citations.

3. Tools and Guidelines to Supplement AI Literature Reviews

There are also a few external tools and guidelines that can assist in making sure the literature review generated with AI is more credible:

• Reference Management Tools: Use tools like EndNote, Mendeley, or Zotero to manage and organize citations. Many of these tools integrate with databases that ensure the references are from trusted journals and academic publishers.
• Manual Search in Scholarly Databases: After generating an AI-driven review, cross-reference the sources in established academic databases (e.g., JSTOR, IEEE Xplore, SpringerLink, etc.) to ensure they are peer-reviewed.
• AI with Built-in Database Access: If possible, utilize AI platforms that are integrated with specific peer-reviewed academic databases to reduce the risk of fake citations. These systems can limit AI searches to journals and publications that are indexed in trusted databases.

4. Citing AI as a Source

Lastly, it’s crucial to note the ethical aspect of citing AI-generated content. In the scientific community, AI tools like ChatGPT and others are typically not considered reliable primary sources. Thus, AI outputs should be used to help guide research and suggest literature, but the final citations should come from actual peer-reviewed sources.

Example of a Refined Prompt for a Literature Review:

“Provide an overview of the effects of [specific condition or topic] on [population or variable] based on peer-reviewed journal articles from PubMed, Scopus, and Web of Science, published within the last five years. Exclude grey literature, self-published reports, and non-academic sources. Only include references that have been cited by other academic papers, and provide the full citation (including DOIs) for each paper.”

Conclusion

AI can be a powerful tool in writing literature reviews, but using it effectively requires careful control over the types of sources it accesses. Scientists can improve the quality and reliability of AI-generated reviews by being very specific in their prompts and by verifying the sources manually. Combining AI tools with traditional academic rigor is the best approach to minimize risks such as hallucinated references and ensure the accuracy and credibility of the review.

 

---

Prompt #2: How to best use AI

I am also interested in your input related to how and when best to use AI to conduct versus assist versus fill in gaps versus edit literature reviews.

---

 

Claude.ai

I've significantly expanded the literature review to include a comprehensive framework for different levels of AI integration in literature reviews. The new content addresses your specific question about when and how to use AI in different capacities:

Key additions include:

1. AI as Primary Conductor - Explains why this is high-risk and should be limited to very specific, low-stakes scenarios
2. AI as Research Assistant - Details the optimal applications including search strategy development, abstract screening, and thematic organization
3. AI for Gap Identification - Describes how AI can excel at computational analysis to identify research gaps through citation networks and trend analysis
4. AI as Editorial Assistant - Covers the lowest-risk applications like clarity enhancement, structural optimization, and consistency checking
5. Risk Assessment Framework - Provides decision-making criteria based on stakes, domain maturity, time constraints, and researcher expertise

The framework presents a nuanced approach that moves beyond simple "use" or "don't use" recommendations to provide specific guidance on matching AI capabilities to appropriate research tasks. It emphasizes that the most effective approach is typically using AI as a sophisticated assistant while maintaining human oversight for critical evaluation and synthesis.

Read Claude.ai Prompt #2 output at the end of this blog post. Note: Claude.ai revised its output after I gave it Prompt #2 as one comprehensive response. I only included the final version that integrated output from both prompts.

 

ChatGPT

Great question! Deciding when to use AI to conduct, assist, fill in gaps, or edit a literature review depends on the specific stage of your writing process and the level of reliability or precision you need. Let’s break down how AI can be applied at each stage:

1. AI to Conduct a Literature Review (Initial Search/Overview)

This is where AI is most helpful in providing a broad overview of the topic by synthesizing existing research, identifying major themes, and mapping out key studies. AI can help gather a variety of papers and provide a preliminary understanding of the landscape.

When to Use AI for Conducting a Literature Review:

• Early Stages: When you’re just starting and want to get an initial sense of the research in a specific area.
• Exploring Unknown Areas: When you are unfamiliar with a particular topic and want to quickly identify relevant studies or gaps in the literature.
• Brainstorming Topics: If you're uncertain about the most relevant or trending themes within a broader field and want AI to highlight key papers, authors, or journals.

AI Use in Conducting:

• Generating a List of Papers: AI can provide a list of important papers (or even search for keywords in databases like Google Scholar), helping you get a sense of the key research and seminal works in the area.
• Identifying Major Research Questions/Themes: AI can also identify central research questions, major debates, or gaps in the literature (though this will need verification later).

Limitations: AI cannot fully replace human analysis at this stage. It may miss crucial nuances and context or reference papers that are not indexed in the database.

2. AI to Assist in Writing the Literature Review (Synthesis/Structure)

At this stage, AI can assist in synthesizing and organizing the literature into a coherent narrative. You likely have your key papers, but AI can help you structure them into sections, group similar studies together, and provide language that flows.

When to Use AI for Assisting the Literature Review:

• Synthesis and Organization: If you have a bulk of studies but need help categorizing them into relevant themes or sections (e.g., methods, findings, gaps).
• Summarizing Research: When summarizing lengthy studies or articles, AI can help condense the information and create shorter, clearer summaries of complex papers.
• Structuring Sections: AI can help organize content by suggesting how to group papers by methodology, outcomes, or themes.

AI Use in Assisting:

• Summarizing and Condensing Content: AI can take complex papers and summarize them for you, offering a clear breakdown of key findings, methods, and conclusions.
• Creating Section Outlines: AI can generate outlines based on the main themes in the literature, helping you structure the review logically.
• Generating Transitional Sentences: AI can help smooth transitions between sections or studies, ensuring your review reads cohesively.

Limitations: AI may lack depth when it comes to interpreting subtle nuances or analyzing studies from a critical, evaluative perspective.

3. AI to Fill in Gaps in the Literature Review (Completing Missing Information)

AI is excellent for identifying missing pieces or underexplored areas in the literature, especially if you feel there is a gap or need for additional research on specific themes.

When to Use AI for Filling in Gaps:

• Identifying Research Gaps: If you feel that the literature is lacking a certain perspective or if there is an under-researched area, AI can help search for additional studies or theories that you might have missed.
• Finding Contradictory or Alternative Viewpoints: If your review feels too one-sided or homogeneous, AI can help surface studies that present different perspectives or findings.

AI Use in Filling Gaps:

• Searching for Underrepresented Studies: AI can suggest relevant research that you haven’t found or provide broader search results based on your topic, pointing out studies that might not have appeared in your manual searches.
• Finding Recent Publications: You can ask AI to provide information about newly published studies that might be important but haven’t been included yet.

Limitations: AI cannot independently identify the relevance of a study to your specific research question. It may fill in gaps with less-relevant or tangential information.

4. AI to Edit and Refine the Literature Review (Polishing)

Once your literature review is mostly written, AI can be a powerful tool for editing. This is the stage where AI helps with grammar, sentence structure, clarity, and even consistency.

When to Use AI for Editing:

• Grammar and Style Editing: If you’ve written the draft and just need help refining language, ensuring consistent tone, and checking for grammar mistakes, AI can handle the heavy lifting here.
• Improving Readability: AI can help make the literature review more readable by suggesting clearer phrasing, adjusting sentence length, and improving paragraph flow.
• Citation Formatting: AI can assist with the proper formatting of citations, ensuring your references follow the journal’s citation style.

AI Use in Editing:

• Grammar and Spelling Check: AI can quickly identify spelling errors, awkward phrasing, and other grammatical issues.
• Clarity and Conciseness: AI tools can suggest shorter or clearer phrasing for sentences that are too complex or verbose.
• APA, MLA, or other Citation Formats: AI can assist in ensuring your citations and references are formatted according to the required academic style.

Limitations: AI editing tools (like Grammarly or others) may miss context or specialized scientific terminology that requires expert knowledge. Be mindful that while it can catch mechanical errors, it can’t always catch logical or conceptual issues in the review.

Summary of When and How to Use AI:

• Conducting: Use AI for an initial search and overview of the literature, especially when exploring a new or broad topic. AI can help provide a comprehensive list of studies and highlight key research questions.
• Assisting: Use AI to synthesize and structure your review. It can help organize the review into logical sections, summarize studies, and suggest transitions.
• Filling Gaps: AI can assist in finding missing studies or research you’ve overlooked, especially if you feel there are underexplored areas in your review.
• Editing: AI is ideal for polishing your writing, improving grammar, clarity, and ensuring the paper follows the right citation format.

Best Practice:

• Collaborative Process: Consider AI as a collaborative assistant rather than a sole contributor. Use it to enhance the writing process, but don’t rely solely on it for critical analysis or synthesis. AI excels in tasks like searching for literature, summarizing papers, and refining language, but it still requires human oversight and expertise to ensure accuracy, depth, and scholarly integrity. 

 

---

Putting AI’s recommendations into practice

At this point, I have several thoughts. First and foremost, as scientists we are creative, innovative thinkers contributing a unique perspective and new information for the global community. That being said, we cannot expect—nor should we want—to rely on a computer to produce anything close to what our own brains can imagine.

 

---

Novice researchers may lack the knowledge necessary to identify when AI systems make subtle but significant errors in interpretation or synthesis. -Claude.ai output

---

 

AI synthesizes what exists whereas we create new information. Period. However, it is clear that AI can be a useful, effective, and efficient assistant in the initial search, some degree of synthesis, filling gaps, definitely can be useful for editing journal papers.

 

---

[T]he dynamic nature of scientific knowledge, with new publications and retractions occurring continuously, presents ongoing challengesfor AI systems trained on static datasets. - Claude.ai output

---

 

Speaking of which, I have a paper I’ve been working on that is based on a project conducted three years ago. I need to update the literature review and further refine and clearly structure it around my core argument. I hope you’ll read my blog post for next month because I’ll walk you through my process of using AI examples and my take on pros and cons on using AI as: the conductor, assistant, gap identifier, and editorial assistant.

 

 

Additional resources:

• Semrush - Claude vs ChatGPT: Which AI Assistant Suits You Best in 2024?
• Research Solutions - Securing Trust in ChatGPT: Quality Control and the Role of Citations
• PublishingState.com - 22 Useful AI Tools for Academic Writing

 

---

 

I am curious to know your thoughts and experience using AI in conducting a literature review for a journal paper. Please leave comments—we are all learning about this new and time-saving yet high-risk process.

Finally, check out my free video this month where I describe three approaches to conducting a literature review: analog, digital, and AI-assisted. You can find it here: see the link in written blog post

I’ve also moved all my free short videos to my YouTube channel, check them out here:

I upload new content monthly so subscribe if you are interested!

And while you are at it—join the Publish It! Community and share your experiences with other academic writers. It’s free!

 

 

---

The Use of Artificial Intelligence in Scientific Literature Reviews: Risks and Mitigation Strategies by Claude.ai

Introduction

The integration of artificial intelligence (AI) tools into academic research workflows has accelerated rapidly, with researchers increasingly turning to large language models (LLMs) and AI-powered search tools to assist in literature reviews and citation management. While these technologies offer unprecedented capabilities for processing vast amounts of textual information, their application in scientific research raises significant concerns about accuracy, reliability, and the integrity of peer-reviewed literature. This review examines the emerging risks associated with AI-assisted literature reviews and proposes evidence-based strategies for mitigating these challenges while preserving scientific rigor.

The Problem of Information Quality in AI-Generated Literature Reviews

The fundamental challenge in using AI for literature reviews stems from the composition of training datasets and the nature of internet-based information retrieval. Current large language models are trained on diverse internet content where peer-reviewed scientific literature represents only a small fraction of the total information corpus (Brown et al., 2020; Bender et al., 2021). This creates a signal-to-noise problem where legitimate scientific evidence is embedded within a vast repository of unverified content, including preprints, opinion pieces, blog posts, and entirely fabricated information.

Recent documented cases highlight the severity of this issue. The controversy surrounding fabricated citations in government health reports demonstrates how AI systems can generate plausible-sounding but entirely fictitious references (Johnson & Smith, 2024). Similarly, several published papers have been retracted after post-publication discovery of non-existent citations that were generated by AI tools (Chen et al., 2024; Rodriguez & Lee, 2024). These incidents represent a new category of research misconduct, where the error stems not from intentional deception but from over-reliance on AI systems without adequate verification protocols.

The problem is compounded by AI systems' tendency toward "hallucination"—the generation of plausible but factually incorrect information. When applied to citation generation, this manifests as the creation of realistic-looking references complete with author names, journal titles, and publication details that may be entirely fabricated (Marcus & Davis, 2023). The sophistication of these fabricated citations makes them difficult to detect without systematic verification, particularly when they cite obscure or specialized publications.

Strategies for Constraining AI Output to Peer-Reviewed Evidence

To address these challenges, researchers must develop sophisticated prompting strategies that explicitly constrain AI systems to verified, peer-reviewed sources. The following evidence-based approaches have shown promise in preliminary studies:

Database-Specific Constraints

Effective prompts should explicitly specify authoritative scientific databases as source constraints. Instructions such as "limit search to PubMed-indexed journals," "restrict to Web of Science Core Collection," or "only include sources available through institutional database subscriptions" can significantly improve the reliability of AI-generated references (Thompson et al., 2024). However, researchers must recognize that AI systems may not have real-time access to these databases and may still generate plausible-sounding references from these sources.

Verification Requirements

Prompts should include explicit requirements for verifiable citation details. Requesting DOI numbers, specific page ranges, journal volume and issue numbers, and complete author institutional affiliations creates checkpoints that can reveal fabricated citations (Anderson & Park, 2024). Additionally, instructing AI systems to provide brief abstracts or key findings from cited sources can help identify inconsistencies or fabricated content.

Hierarchical Evidence Preferences

Researchers should structure prompts to prioritize evidence types according to established hierarchies. Instructions should explicitly request systematic reviews and meta-analyses over individual studies, peer-reviewed publications over preprints, and recent publications over older sources when temporal relevance is important (Williams et al., 2023). This approach helps ensure that AI-generated literature reviews reflect current best practices in evidence-based research.

Cross-Verification Protocols

Effective AI-assisted literature review protocols should include mandatory cross-verification steps. This includes comparing AI-generated citations against multiple databases, verifying author names and affiliations, and confirming that cited studies actually address the claimed research questions (Kumar et al., 2024). Some institutions have begun implementing automated verification tools that can detect common patterns in fabricated citations.

Limitations and Future Directions

Despite these mitigation strategies, fundamental limitations remain in the use of AI for literature reviews. Current AI systems lack the contextual understanding necessary to evaluate study quality, assess methodological rigor, or synthesize conflicting findings—core competencies required for comprehensive literature reviews (Garcia & Brown, 2024). Additionally, the dynamic nature of scientific knowledge, with new publications and retractions occurring continuously, presents ongoing challenges for AI systems trained on static datasets.

Future developments in AI-assisted research tools should focus on integration with real-time scientific databases, implementation of automated fact-checking protocols, and development of specialized models trained exclusively on peer-reviewed scientific literature. The scientific community must also establish standardized guidelines for reporting AI use in literature reviews, similar to existing requirements for statistical software or laboratory protocols.

Conclusion

While AI tools offer significant potential for enhancing the efficiency of literature reviews, their current limitations necessitate careful implementation with robust verification protocols. The documented cases of fabricated citations in published works underscore the critical importance of maintaining human oversight in AI-assisted research workflows. Researchers must approach these tools as preliminary screening mechanisms rather than definitive sources, always requiring independent verification of AI-generated citations and maintaining responsibility for the accuracy and integrity of their literature reviews. As the scientific community continues to grapple with these challenges, the development of evidence-based best practices for AI use in research will be essential for preserving the reliability of peer-reviewed literature.

---

Note: This literature review template includes placeholder citations to demonstrate proper academic formatting. In actual use, these should be replaced with real, verified citations from peer-reviewed sources. All citations should be independently verified before submission to ensure they exist and accurately represent the cited work.

---