By Invitation Only: ‘We believe you have the expertise…’
Apr 01, 2025
Hurray you're invited!
Receiving an invitation from a journal editor to peer review a manuscript is both exciting and annoying. At least that is my experience. I get excited because I’ll be privy to cutting edge as-yet-unpublished research that the editor has decided I am uniquely qualified to determine its validity, significance, and originality. The email is proof of my expertise.
At the tail end of my jolt of self-satisfaction comes the twinge of overwhelm when I glance at my full calendar. A meaningful review will cost me three to four hours of time—concurrent hours; a block of hours. While it is less than half of the time needed when I started reviewing papers as a new graduate, it’s still a substantial chunk of time. I don’t have that kind of time. And it’s not like I’m getting paid—peer review is on a voluntary basis. Plus, if the manuscript is poorly written, I’ll have to expend a lot of energy identifying the purpose of the paper and giving detailed feedback.
Peer Review is a Gateway
Before I talk myself out of accepting, I remind myself of the critical role of peer review. With the deluge of information—human and AI created—peer review has become an ever more important gateway for knowledge sharing. By ‘information’, I mean all the stuff out there including words and sentences and data. By ‘knowledge’, I mean how we make sense of information through scientifically rigorous methods.
Peer review is the process by which experts in the field evaluate scientific and academic papers before publication to ensure high quality, accuracy, and integrity. In other words, peer reviewers are typically experts in the same field as the research paper they are asked to review--often academic researchers or professionals who understand the subject matter, methodology, and current trends. Journal editors usually identify potential reviewers based on their previous work, reputation in the field, and prior experience with peer review. The selection of peer reviewers is a critical step in the review process.
Improving Quality & Detecting Problems
Peer review improves research quality by:
- Identifying flaws in methodology, analysis, and interpretation
- Detecting errors, inconsistencies, and gaps that need clarification
- Providing constructive feedback for improving rationale, discussion, etc.
- Screening for plagiarism and conflicts of interest
- Motivating researchers to produce more rigorous, unbiased work
Peer review can also detect problems:
- Duplicated images suggesting data fabrication/manipulation
- Misinterpretation of statistical analysis undermining conclusions
- Inadequate methodology or analysis to support conclusions
- Failure to identify limitations or consider alternative explanations
- Plagiarism or inappropriate text reuse
Reviewers are trusted with confidential material--this means you can’t share it with anyone without prior authorization from the editor, and you definitely cannot cite it, reference it, or integrate any part of it in your own research. Actually, there are a number of ethical considerations. Check out the Committee on Publication Ethics (COPE).
Types of Peer Review
|
Double blind |
Single Blind |
Open |
|
The reviewers and authors are both anonymous to each other throughout the process. |
The reviewers know the names and affiliations of the authors, but the authors don’t know who the reviewers are. |
Reviewers and authors are known to each other. Reviews may be published alongside the paper. |
|
Double-blind is considered the gold standard, as it minimizes bias. However, a completely anonymous review can be difficult for niche topics where authors are easily identifiable. |
Bias can occur when reviewers are influenced by the author's reputation, institution, gender, or nationality. Authors from prestigious institutions or with established reputations can have an unfair advantage. |
Increases transparency of the authors and accountability of the reviewers. Bias can occur when reviewers are influenced by the author’s reputation, affiliation, or past work, potentially leading to preferential treatment. |
Should you use AI?
With the advancement of AI technologies, there is a lot of discussion around its use in writing and reviewing journal papers. That is fodder for a future blog, but if you are interested in reading more, check out.
Guidelines for Peer Review
Every journal has its own guidelines for peer review—you should always check that first. Every publisher also has guidelines, for example Elsevier has guidelines and tutorials.
That being said, here is a template that I use when I review papers:
- Journal Title:
- Scope/Overview:
- Is the subject of the article within the scope of the journal?
- Does the title clearly and sufficiently reflect its content?
- What does the paper set out to do, and is it successful?
- Are the interpretations/conclusions sound and justified by the data and analysis?
- Are the illustrations, tables, and references adequate, necessary, and acceptable in their current form?
- Is the language appropriately non-technical for a multi-disciplinary readership?
- Is the paper interesting, important, innovative, adequately rigorous, and generally well-written?
- Specific items:
- Overall, this paper: [State the potential contribution and broad strengths and weaknesses]
Peer Review Helps Reviewers
In conclusion, peer review is a service experts provide directly to the scientific community and indirectly to the global community. But experts also benefit by learning about the newest research—including new methods, approaches to theory, analytical tools, interpretations—and, yes, even how to write better papers by example or avoidance of bad habits. I have found it immensely useful in the evolution of my own scientific writing to see early manuscripts develop into strong published papers.
One last thought on peer review that I will leave you with—and this is something that has taken me years to adopt—is to set a limit on the number of peer reviews you agree to each month. With the vast number of journals coming online in recent years and their need to publish more papers and faster, it’s easy to be seduced by the multiple editors vying for your expertise and time. Just remember, while reviewing and writing papers go hand in hand, they are not equal on your CV.
Author
