Triage: What a Tired Reviewer Sees in Your First Three Pages

About an 18-minute read, or listen to the audio version (about 30 minutes).

In this post: The Inbox · The Waiting Room · The Symptom Checker (my AI experiment) · Whose Symptoms Get Believed · What 88 Reviews Say · Stern, Never Harsh · Left Against Medical Advice The Cartilage Report

The Inbox

A reviewer usually decides whether to read your paper from its title and abstract alone, in about a minute, in whatever kind of week they're having.

Peer Review Week opened on September 14th. This year's theme was capacity.

That week, four invitations to review landed in my inbox. I declined all four.

I had reasons. A deadline of my own. A topic just far enough outside my expertise that I'd have to be transparent about it. A journal I'd already reviewed for twice this year. Each decline took less than a minute.

The next week, my Scopus access expired. Leaving me, someone outside the academic tower, back behind a paywall. The timing was not lost on me.

Here is what I had in front of me each time: a title, an abstract, a deadline. No methods. No figures. No discussion section that someone had rewritten six times. Four research teams had spent years on those papers, and I decided in under a minute that I would not be the one to read them.

In an emergency room, this is called triage: the first minute a nurse spends deciding who gets seen, and how soon. Nobody has looked at the fracture yet. It's a fast judgment made from whatever is visible at the door, by someone with more patients than time.

If you have submitted a paper, someone has triaged it. Probably several someones, before one finally said yes. And the one who said yes made the same calculation I did, from the same 200-word paragraph, somewhere between their own deadlines and whatever else was on fire that week.

So what did your abstract tell them?

The Waiting Room

Editors now send about twice as many invitations to get one completed review as they did fifteen years ago, so a slow or thin review of your paper is often a capacity problem first.

I was not the only one saying no.

In 2013, an editor needed to send an average of 1.9 review invitations to get one review report back. By 2017 it was 2.4, and the share of invited reviewers who agreed had fallen from about 54% to 44% (Publons, 2018). One journal has tracked this for nearly two decades. In 2007, its editors needed four invitations to secure two reviews. In 2025, they needed eight (Estuaries and Coasts, 2026). The waiting room is full, and the staff keeps shrinking.

On your end, this shows up as months between submission and a decision. As a review that seems like it was typed on a phone between meetings. As comments that make it obvious someone skimmed your methods. Some of that is about your paper. A lot of it is about the evening your reviewer had.

The intake form

When an editor invites me to review, the email holds your title, your abstract, and a deadline. That is the whole intake form. From it, I decide whether the paper sits inside my expertise, whether I can give it the hours, and whether I'll have something useful to say.

So your abstract has to carry everything a stranger needs to say yes: the gap, your core argument, your key finding, and why it matters. Your core argument is the one claim your paper exists to make. If it only appears in your discussion, the reviewer deciding whether to read your paper will never see it.

I hate saying you have to sell your paper to a reviewer. But there, I said it.

Why I still say yes

Nobody pays me to review. You're gonna say none of us get paid to review. True. But inside the university, my salary covered those hours and they counted as service. Now I review because it keeps me current in my field, and because it shows me, in real time, what's getting published and rejected, and why.

I also say yes because peer review is how we ensure scientific rigor. When qualified reviewers stop agreeing, rigor doesn't collapse all at once. It thins. Papers get read by whoever is available instead of whoever knows the work.

How I decide, in this order: expertise fit. My own deadlines (I rarely take on more than two reviews in a month). How often I've reviewed for that journal recently (are they treating me like an unpaid employee?). Finally, and this might get pushback, how interesting the abstract makes the paper sound: its innovation, its novelty, whether the authors sound like they even like doing this research or whether it's just work. You can pick any topic as a scientist. Why wouldn't you pick what you're passionate about? Me? I love food. Check my CV.

How long a review should take

A careful review takes me about three hours, in one block. It didn't start that way. Early on, I blocked four hours one day and two more the next, just to turn my notes into something authors could use. What changed is that I know where to look now. More on that later.

If a review of your paper came back in a week with just three lines, somebody didn't have the three hours. Keep that in mind when you decide how much weight to give it.

The Symptom Checker

Publishers are still writing the rules for AI in peer review, many reviewers use it anyway, and when I tested it on my own paper, it found problems my human reviewers missed. It still couldn't tell me what to do next.

Most of us now type our symptoms into a phone before we ever see a nurse. It's fast. It's available at 2 a.m. And lately, it's uncomfortably good at spotting the fracture.

Peer review has its own symptom checker now.

The rules, so far

Elsevier's policy, updated in June 2026, says reviewers "should not upload a submitted manuscript or any part of it into an AI tool," because it may violate the authors' confidentiality (Elsevier). Other publishers draw the line elsewhere, and a global standard for what everyone should disclose is still in consultation (COPE).

So researchers are already being held to disclosure rules that are still being written. By people who will also be reviewing them.

What reviewers actually do

Machine learning is not our field, but it's the field that ran the experiment. At ICML 2026, a major machine learning conference, organizers randomly assigned roughly 17,000 reviewers to one of two policies: no AI, or limited AI use. Among reviewers told not to use AI, 22.5% reported using it anyway. The policy made almost no difference to paper scores, final decisions, or reviewer confidence. Its clearest effect was that reviews written under the permissive policy ran 5.5 to 7% longer (Kim et al., 2026).

Rules like these mostly bind the people who were already careful. Assume your reviewer may be using AI, whatever the journal says.

I ran the experiment on myself: human vs. AI peer review

I took one of my own papers: the first-submission draft of a single-authored paper I later published in an MDPI journal. I gave it to Claude and to ChatGPT, twice each, in fresh chats, with the same neutral prompt: You are an experienced peer reviewer for a peer-reviewed journal in urban food systems. The editor has asked you to review the attached manuscript. Write a review report as you would submit it: a brief summary, your major concerns, your minor concerns, and your recommendation, with the main reasons for it. I turned off memory and web search. Like my original reviewers, the models saw my name; the journal uses single-blind review. The paper is open access, so the models may have met it before.

Then I read four reviews of my own paper back to back. I have no shame in reporting that I did not take the review output well.

For context: I wrote that paper in two weeks. I was aiming for a major revision, and I got one. All four AI reviews recommended major revision too. That's where the resemblance ends.

Two human reviews and four AI reviews of the same paper, compared on length, recommendation, and what each one caught.

Here is the uncomfortable part. The machines found the fracture. All four caught structural problems, the kind that decide a recommendation, and neither human reviewer named them. All four also caught a units error in my Table 2 that defined a "small" farm as anything under 100 square meters. Most of the farms in my study were measured in acres. My reviewers missed it. So did I.

What the machines did miss was everything that required knowing the place. And one Claude review faulted my 2019 fieldwork for not discussing a pandemic that hadn't happened yet.

So what do you do with it?

A large language model (LLM), the technology behind tools like Claude and ChatGPT, will review your paper at 2 a.m. in under five minutes. The harder question is what you do with 2,000 words of concerns at 2:05.

It doesn't prioritize. Each review the LLM spat out to me listed twenty-some problems, with no sense of which two or three would decide the recommendation versus which were merely an afternoon of housekeeping.

It doesn't know where you're trying to go. It reviews toward the paper it would write. One of my Claude reviews ended by suggesting that a better paper would be organized around migrant labor and "who the networks cost." Maybe. But that was never the paper I set out to write, and it certainly wasn't a two-week paper aimed at an R&R (revise and resubmit).

And it has never done the work. These models are intelligent in a real sense. They hold more literature than any reviewer I know. What they don't have is experience: years in the field, in the reviewer's chair, in the room where a paper goes from rejected to published. They recognize patterns. They can't tell you which pattern matters for your paper, your journal, and what you're trying to say.

Whether you use it is your call. It isn't my recommendation, but it is an option, as long as you use it responsibly. That means your own unpublished work only, never a manuscript you've been asked to review. Check your target journal's AI policy first. Get your co-authors' agreement before uploading shared work. And turn off any setting that lets the tool keep or train on what you give it.

If you do use AI, be ready for what comes back: a long, confident list with everything weighted the same. Sort it before you touch your draft. Which items are about your argument, your structure, your interpretation? Those are the ones a reviewer will decide on. Which are housekeeping? And which would pull you toward a paper you never meant to write? Leave those on the list.

A list is not a diagnosis.

Whose Symptoms Get Believed

Suspicion that a paper was written with AI falls hardest on careful writers working in a second language, and it pulls reviewers away from the question that decides a recommendation: is there an argument?

Triage has a known flaw. The nurse at the door makes fast judgments, and fast judgments carry bias. Medicine has documented for years whose pain gets believed and whose gets discounted (Hoffman et al., 2016).

Peer review has its own version. This year it has a new symptom: writing that sounds too smooth.

A recent piece in Nature Machine Intelligence argues that the growing climate of AI suspicion in publishing may now do more damage than undisclosed AI use itself. The damage looks like false accusations, and a chill on clean, competent writing that happens to appear a little too polished (Nature Machine Intelligence, 2026).

Think about who produces that writing. It's a researcher working in their second or third language, who has checked every article and every preposition. Maybe they paid for editing. Maybe they used a grammar tool the journal explicitly allows. Careful writing is exactly what an unsure reviewer flags.

I know, because I have been that reviewer.

When I went back through my reviews for this post, AI didn't come up once before 2025. In 2026, I raised it in four of about a dozen. Twice, I asked the authors directly whether and how they had used AI. Twice, I went further and wrote that the sentence patterns didn't match what I usually see from a particular group of EFL authors that I review regularly.

This year I also flagged a paper to an editor as possibly AI-written. I still submitted a full review. I asked the authors about AI use, and they said they hadn't used it. I'm still skeptical. I also know my skepticism rests on exactly the kind of pattern-matching the Nature Machine Intelligence piece warns about.

Here's what is clear to me. A well-written paper cannot hide a missing argument. Period. Full stop. Smooth sentences, whoever polished them, don't make a gap appear, don't hold a structure together, and don't make a discussion interpret its results. Once I strip a paper back to its argument, who polished the sentences stops mattering to my recommendation.

So why do reviewers, including me, reach for the AI question at all?

Did anyone teach you how to write a journal paper? Me neither. Nobody taught us how to review one, either. Without a system, reviewers fall back on what is easiest to see, and the surface of the writing is the easiest thing to see. Suspicion fills the space where a method should be.

So what does a reviewer look for when they do have a system? I went back through twelve years of my own reviews to find out.

What 88 Reviews Say

Across twelve years of my review reports, grammar was never the deciding factor in my recommendation. Instead, it was whether I could find the argument, whether the structure carried it, and whether the discussion interpreted the results or just repeated them.

A few years ago I wrote a list of 13 reasons papers get rejected. Lists are useful (this one was truly based on my personal experience). But I wrote it off the cuff rather than as a systematic synthesis.

So for this post, I did something I'd never done. I went back through 88 of my own review reports (from a current 103 across 46 journals; my records are incomplete). These were written between 2014 and 2026 for journals in urban agriculture, food systems, planning, public health, geography, and urban forestry. I counted what I actually said. The counts are my own coding, so treat them as approximate. The patterns are not subtle.

Grammar is the last line

Grammar and syntax came up in about three of every ten reviews, almost always in the final sentence: a thorough copy-edit is needed. It never decided a recommendation. In contrast: I recommended rejecting one paper in the same review where I called it "fairly well-written."

What came up far more often:

• A weak or missing discussion: about 8 in 10 reviews
• Methods too thin to evaluate or repeat: about 8 in 10
• No clear aim or core argument: about 7 in 10
• A gap that was named but never justified: about 7 in 10
• Claims that over-reached the evidence: about half

Every one of these is structural. None of them can be fixed with a grammar tool.

A report where a paper should be

One comment has grown faster than any other in my reviews: this is written as a report, not a journal paper. I made it in 1 of 18 (6%) reviews from 2014 to 2019, 8 of 40 (20%) from 2020 to 2023, and 14 of 31 (45%) from 2024 to 2026. (A note on reliability: somewhere around 2016, I learned how essential a core argument is for a journal paper, so I started reading for it specifically.)

The difference is simple to state and hard to fix. A report summarizes all the data. A journal paper selects the evidence that supports one argument and tells the reader what it means. Or, as I put it to one set of authors: examples are not findings. They are the data.

Three tests I run without meaning to

When I read your paper, I'm running three tests, whether or not I've named them. They map directly onto the recommendation you'll get back.

Test 1: Can a tired reader find your argument? After your abstract and introduction, I should be able to write your core argument in one sentence. When I can, I often do, right in my review. When I can't, the review says so: what is the core argument? A gap alone isn't an argument, either. I've written this sentence in reviews for twelve years: just because a gap exists doesn't mean it needs to be filled. Tell me why it matters.

Test 2: Does your structure carry the argument? The organizational logic should hold from introduction to discussion: the question your introduction raises is the question your methods answer, your results report, and your discussion interprets. The most common break I see is a paper trying to be two papers, a methods paper and an empirical case study at once. It can't be both. Two strong, focused papers are better than one ambiguous, weaker one.

Test 3: Does your discussion interpret, or just repeat? A discussion should say what the results mean, relate them to the literature, question them, and name their limits. It should also stay inside what the study can claim. Cross-sectional data shows correlation, not cause, and quantitative data can tell you how and how much, but rarely why.

Last month I saw the strangest version of test 3 I've seen. It was a paper that had already been through two rounds of review. Not two reviewers. Two rounds. I was the new reviewer. The authors reported their main result with a tone of implied disappointment: no meaningful effect. I read their results table three times. Their own data showed a significant positive effect. They had misread which direction their scale ran. I remember thinking (imposter syndrome alert!): Am I crazy? How am I seeing something everyone else missed? Twice? My review said it plainly: this changes the whole paper.

Reject, major, or minor revision: where the three tests land you

Minor revision means the editor will accept the paper after small changes, often without another full round of review. In my reviews, these papers pass all three tests. I can state the argument back, and what's left is housekeeping: limitations, statistical reporting, figures, tense, the copy-edit.

Major revision means the paper might be publishable after substantial changes and another round of review. About nine in ten of my first-round recommendations land here. The research is often sound. The paper fails test 1 or 3, or both, and sometimes test 2. Everything a major revision asks for can be fixed by rewriting.

Reject means the journal won't consider this version again. I recommend it rarely, fewer than five times in the reviews I went back through. It happens when test 2 fails at the level of the study itself: the research can't answer the question the paper asks, and no amount of rewriting will change that. In one review, I wrote that the paper had "methodological flaws that cannot be improved by writing." Ethics problems land here too.

Notice what's missing from all three tiers. Nowhere does the writing's surface decide where a paper lands.

Stern, Never Harsh

Reviewing other people's papers is one of the fastest ways to learn to write your own, and a review you receive is best read as a map of where one careful reader got lost.

My earliest reviews followed the journal's form, question by question. Is the subject within the scope of the journal? Does the title reflect its content? I answered each one dutifully, because I didn't have a system of my own. Those reviews took two days.

Somewhere along the way, the form stopped being enough. Around 2020, I started opening every review with the three main concerns, then the details. By 2024, the same discussion checklist appeared in almost every review I wrote: say whether the hypothesis was confirmed, link the results, relate them to the literature, name the implications, claim the significance, question the findings, note the limitations, suggest future research.

This year, reviewing for a humanities journal, I caught myself writing a new set of questions at the top of the document. Is there a legible central claim, and does the paper keep returning to it? Is the theory doing analytical work, or is it decorative? Does the conclusion do something, or does it just stop?

Nobody handed me that system. I built it one paper at a time, mostly from other people's mistakes. Then I started catching the same mistakes in my own drafts. That's how six hours became three.

This is the part nobody tells early-career researchers about reviewing. It counts as service, and it's also training. Every paper you review shows you, from the reader's chair, exactly where a paper loses someone. If an editor invites you and the paper sits inside your expertise, say yes to the first few. And when a review of your own paper lands, read it the same way: every comment marks a place where one reader got lost, stalled, or stopped trusting you. Some reviewers mark those places kindly. Some don't. The mark is useful either way.

Patience with a messy paper

I still get papers that are hard to read. Sections in the wrong place. Findings in the introduction. A discussion that turns out to be a second literature review. When I feel my patience going, I remember that form I used to fill in, question by question. Nobody taught these authors, either.

So I try to review the way I'd want to be reviewed: stern about the paper, never harsh with the people who wrote it. When I can see the argument buried in someone's discussion, I pull it out and hand it back. When a paper is trying to be two papers, I say which one I'd write first. Three times, I've recommended the same book to authors, with the same note: when I was early in my writing career, this book helped me immensely (Wendy Belcher's Writing Your Journal Article in Twelve Weeks).

Left Against Medical Advice

Many papers that clear the hardest hurdle in publishing, being sent out for review and invited to resubmit, never come back.

Here is a number I didn't expect when I went back through my reviews: how rarely I see a paper a second time.

I recommend major revision on about nine in ten papers in the first round. That's an invitation to resubmit. It means an editor and at least two reviewers read the paper and decided it was worth another look. Anyone who has collected a desk rejection knows how special that is. It's the biggest hurdle in the whole process.

And yet I review the revised version only occasionally. Some of those papers probably go to new reviewers. But editors are struggling to find reviewers at all, and I write long, specific, actionable comments. The more likely explanation is simpler: the authors never resubmit.

Earlier this year, I reviewed a paper from a research team in South America, from a region that rarely shows up in the journals I review for. The research was innovative and sound. The paper was trying to do too much: it was a methods paper and an empirical case study at once. I laid out two clear paths: pick one lane, resubmit, and save the other for a second paper.

A week later, the editor emailed me. The authors had withdrawn the paper.

In an emergency room, there's a term for this: left against medical advice. The patient was seen. The diagnosis was clear. The treatment plan was written down. And they walked out anyway.

I was bummed. Truly. I don't know why they left. Maybe the revision looked bigger than it was. Maybe a major revision sounded like a polite rejection. Maybe nobody had ever told them that major revision is where most published papers pass through on the road to publication.

I've written diagnoses like that one for twelve years. Unpaid, anonymous, and always after submission, when a paper has already spent months in the waiting room and the authors have already braced for bad news.

So who reads your paper carefully before it counts?

Before the Waiting Room

An AI can list what's wrong with your paper in five minutes. The Cartilage Report tells you which problems will decide your recommendation, and what to fix first.

Cartilage is what connects the bones. In a journal paper, it's the argument that connects your data to your claims, and it's the first thing a tired reviewer goes looking for.

Maybe you've already run your paper through an AI and you're staring at a long list. Maybe you'd never do that. Either way, the question is the same: what matters most — what revisions will actually get you over the invitation to revise and resubmit hurdle — and which are merely superficial?

The Cartilage Report answers that. I read your paper the way a reviewer will and tell you which two or three problems will decide your recommendation, which ones are housekeeping, and where your argument is already strong enough to build on. It's two pages, built on more than a hundred reviews and on what you are trying to say.

Get your Cartilage Report Here→

Footnotes

Don’t forget to check out my new videos this month. I upload a new one each Wednesday.

You can find them in the Publish It! Library 

Or watch on the Publish It! YouTube channel. Subscribe if you are interested!

If you are ready to Draft It! check out The Essential 10-Week Workshop – a self-paced course with tutorials, community discussion board, and workbook designed to guide you in preparing your first draft in 10 weeks for submission to a peer reviewed journal! Don’t forget that it starts with a 1-hour intake session with me to lay out your unique road map!  

And while you are at it—join the watering hole, a Publish It! Community where you can share your experiences with other academic writers. It’s free!

References

• Elsevier. Generative AI policies for journals. Updated June 2026.
• Hoffman, K. M., Trawalter, S., Axt, J. R., & Oliver, M. N. (2016). Racial bias in pain assessment and treatment recommendations. PNAS.
• Kim et al. (2026). Use and Effects of LLMs in Peer Review: A Randomized Experiment and Survey at ICML 2026. arXiv.
• Nature Machine Intelligence (2026). On the troubling rise of generative AI suspicion in academic publishing.
• Publons (2018). Global State of Peer Review.
• Estuaries and Coasts data explains why peer review is slow and there is a reviewing crisis (2026). Estuaries and Coasts.
• COPE. Global reporting standard for AI disclosure in research.
• Belcher, W. L. (2019). Writing Your Journal Article in Twelve Weeks (2nd ed.). University of Chicago Press.