The Art of the Review
A practical discussion of the scientific review process from the perspectives of authors, reviewers, and editors.
The central question
Peer review sits at the center of scientific publishing, yet most investigators learn how to review manuscripts informally—through experience, mentorship, and exposure to reviews written by others.
This Research Forum session examined different approaches to the review process and asked what makes a review rigorous, constructive, fair, and genuinely useful to authors and editors.
Central Idea
A good review does more than identify flaws. It helps determine whether the scientific question is important, whether the evidence supports the claims being made, and what would be required to make the work stronger.
The reviewer’s task
Evaluate the science
Is the question meaningful, the design appropriate, the analysis credible, and the interpretation proportional to the evidence?
Separate major from minor
Distinguish issues that threaten validity or interpretation from changes that would merely improve presentation.
Be constructive
Criticism should help the author understand both what is wrong and, where possible, how the work could be improved.
Advise the editor
The review is not only a message to the authors. It is also an assessment for the editor about whether the manuscript is suitable for publication.
A snapshot from the room
A brief audience survey during the session captured how participants experienced the peer-review process. Twelve responses were recorded. These results are descriptive and represent this small group rather than the broader scientific community.
12 / 12 Authors + Reviewers
Every respondent identified as both an author and a reviewer.
5 / 12 Editors
Five participants also reported serving in an editorial role.
6 / 12 ≥10 Invitations
Half reported having been asked to review at least 10 manuscripts.
7 / 12 ≥90 Minutes
Seven respondents reported typically spending at least 90 minutes completing a review.
8 / 12 Support Compensation
Eight respondents indicated that reviewers should receive compensation.
5 / 11 Double-Blind Preferred
Among the 11 who answered, double-blind review was the most frequently selected approach; open and closed review each received three selections.
Review experience and incentives
The survey suggested a group with meaningful experience but mixed views about how the review system should recognize that work.
Review volume
Two respondents reported reviewing at least 10 papers, one reported 7–9, five reported 4–6, and four reported 1–3.
Time commitment
Three reported 60–89 minutes per review, four reported 90–119 minutes, and three reported 120 minutes or more. Only two reported 30–59 minutes.
Incentives
Only 2 of 12 respondents described themselves as currently incentivized to review.
Paid review
Five respondents said reviewers should be paid, while eight supported some form of compensation more broadly.
The Time Burden
In this small group, peer review was rarely a quick task: 10 of 12 respondents reported spending at least an hour on a typical review, and 7 of 12 reported spending 90 minutes or longer.
Why do reviewers say yes?
The survey responses reflected several different motivations for accepting review invitations.
- A desire to improve the field.
- The opportunity to read emerging science.
- A sense of academic duty.
- The chance to improve critical-thinking skills.
- Invitations from colleagues or mentors.
- Feeling honored to be asked.
- Professional recognition and CV development.
The responses reinforce that peer review can simultaneously function as scientific service, intellectual engagement, professional development, and quality control.
Conflicts, anonymity, and trial review
The survey also surfaced some of the less visible judgments reviewers routinely make.
Conflict of interest
Four of 11 respondents with available data answered “yes” to one conflict-of-interest scenario, while seven answered “yes” to a second scenario—illustrating that perceived conflicts can depend heavily on context.
Review model
Preferences were divided: five selected double-blind review, three open review, and three closed review.
Trial review
Four of 11 respondents with available data reported prior experience reviewing a clinical trial manuscript.
Formal training
None of the 12 respondents reported having completed a formal course in peer review.
An Informal Apprenticeship
Everyone in the room had reviewed manuscripts, but no respondent reported formal coursework in how to do it. Peer review remains, for many scientists, a skill learned largely by apprenticeship.
Questions explored
The discussion extended beyond how to critique a manuscript and into the structure of peer review itself.
Open or anonymous?
What are the tradeoffs among open, closed, single-blind, and double-blind approaches to peer review?
Incentives
Should reviewers be compensated, formally recognized, or otherwise rewarded for the time and expertise that peer review requires?
Conflicts of interest
When does scientific proximity become a conflict, and when is subject-matter expertise precisely why a reviewer should be selected?
Accept, revise, or reject?
How should reviewers distinguish work that is fundamentally flawed from work that can become publishable through revision?
Research Forum focus
The session approached peer review as a scientific skill rather than an administrative task. The aim was to make the review process more explicit: how to read a paper critically, how to structure comments, how to calibrate recommendations, and how to provide criticism that is rigorous without becoming performative or unnecessarily adversarial.
For authors, reviewers, and editors alike, the broader goal is the same: improve the quality, clarity, and interpretability of the scientific record.
Audience survey results shown here are descriptive summaries of 12 responses collected during the May 12, 2023 Research Forum session. Some individual survey items had one missing response; denominators are shown where relevant.