Conducting a Manuscript Peer Review

Peer review is vital. Despite flaws, it helps ensure scientific integrity in academic publications. The process benefits the reviewer, sharpening writing skills through exposure to multiple perspectives during the process.¹

Some potential reviewers perceive insufficient competency to peer review, despite available resources (eg, PRiMER,² Family Medicine,³ Association of American Medical Colleges⁴). Here, we offer plain-language advice.

The purpose of peer review is to assess whether a manuscript, if published, would:

• Add valuable, valid information to the relevant body of literature;
• Make sense to the intended audience; and
• Connect with related literature.

Academic literature reflects a conversation, preserved in writing, that advances knowledge. The peer reviewer assesses whether and how a manuscript contributes, and should consider if a manuscript:

• Connects with a conversation (literature review),
• Defines its place in the conversation (adds something new),
• Describes valid processes and outcomes,
• Arrives at reasonable conclusions, and
• Is understandable.

Peer reviews can be lengthy. It is common for reviewer comments on short papers to approximate the length of the manuscript itself. Although longer is not always better, editors often see unhelpful comments like:

“Great paper. Authors use run-on sentences. Otherwise, I recommend publication!”

Most manuscripts are submitted with flaws, points that raise questions, or areas needing improvement. Peer review comments should detail items to consider for revision. Peer reviewers may comment on obvious errors and poorly written or confusing text, however, most journals copy edit for grammar and style. Therefore reviewer comments should focus instead on more substantive improvements.

Each section of a scholarly manuscript should be evaluated individually.

The title and abstract should be descriptive, accurate, and succinctly reflect the manuscript’s content. The abstract should include all key elements of each manuscript section and not introduce different concepts or provide different information. Peer reviewers should revisit the title and abstract after reading the manuscript to ensure alignment and comprehensiveness.

Sometimes called “Background,” the introduction should do three things: provide an entry to the topic, relate to and cite relevant literature, and describe the manuscript’s purpose. A good peer review will comment on presence and clarity of these items.

The methods section should contain a description of all study steps. The EQUATOR network contains standard criteria for assessing manuscripts,⁵ including PRISMA (systematic reviews), STROBE (observational studies), CONSORT (clinical trials), COREQ (qualitative studies), and others. While word limits may constrain descriptions, the methods should provide sufficient detail that others could replicate the study. Typically, a description of an institutional review board (IRB) interaction also belongs in the methods section.

The methods should identify study objectives, target population, sampling and data collection techniques, analytic procedures, and regulatory statements. The methods section should not contain findings, like sample demographics, that more appropriately belong in the results section. Sometimes, it might be appropriate to include the results of instrument validation processes or beta testing in the methods, depending on context.

The results section should contain objective descriptions of observations and data analyses, ideally with minimal to no interpretation, starting with a description of study sample characteristics, followed by all results that emerged from the study. New processes should not be described here (they belong in the methods section) and results of each process introduced in the methods must be included in the results.

Here, peer reviewers should assess for obvious errors, incorrect application of methods, or clear flaws. When uncertain about methodology or content, reviewers may notify the editor, directly in the review, or in confidential comments to the editor (via the appropriate field within the peer review software system). Multiple peer reviewers are selected for varying expertise and perspectives.

The conclusions (or discussion) section is where authors should describe and interpret their main findings and study limitations. Authors may editorialize, however, peer reviewers should raise questions if authors stray from their results, make sweeping generalizations, draw inferences unsupported by study findings, or digress on tangents. Limitations might relate to sampling, study size or site, literature gaps, methodologic challenges, confounding, or generalizability, among others, and should be interpreted in the context of the study itself.

Tables and figures should be useful, legible, identified in the main narrative, and not duplicate information. They should be interpretable by the intended audience without requiring a full reading of the manuscript (eg, include accurate, descriptive titles and clarifying footnotes, when needed).

The manuscript should connect with relevant literature continuing the conversation, citing prior work, results of similar studies, and important methodological references. The introduction and conclusion sections may also cite references supporting the literature gap this study fills, its impact, and suggested next steps. Reviewers should be prepared to offer suggestions of references that were omitted or if more recent/more appropriate references could have been used.

The format of peer review comments may be suggested by the journal’s guidelines. If not, reviewers may consider:

• A section-by-section approach, providing comments on each section sequentially; or
• A point-by-point approach, dividing comments into major and minor concerns. Major concerns may require substantial rewriting, more work, or the authors to address a fundamental flaws. Minor concerns may be addressed by editing or clarifying text.

Sometimes, a short statement might be appropriate. For example, “The introduction and methods seemed to describe different projects; this manuscript requires such a substantial revision that I can’t conduct a proper review.” It helps to add details, however, the reviewer is referring the paper back to the editor.

Authors see peer review comments. Reviewers should be kind, constructive, avoid personal attacks, check for bias, and support authors’ improvements doing revisions and future studies.

Finally, peer reviewers make one of the following recommendations regarding publication:

• Accept: The manuscript is ready to publish with little or no modification.
• Minor Revision: Nearly ready, but needs some clarification, added citations, or editorial improvement. This usually excludes substantial rewrites, additional analyses, or new data collection.
• Major Revision: Needs substantial rewriting, additional data collection or analyses, new tables, etc. Reviewers should recommend this if authors could reasonably quickly complete the recommended revisions.
• Reject and Resubmit: Needs complete rewriting, additional data collection or analyses, new tables, etc. Reviewers should recommend this if the recommended revisions will likely require substantial time or effort or may not be feasible.
• Reject: Should not be published. Contains fundamental methodological flaws, adds nothing to the conversation, contains serious bias, scientific or ethical malfeasance, poor quality writing prevents understanding, or is outside the journal’s scope.

Peer reviewers provide recommendations to journal editors who make decisions about next steps. The journal editor will typically incorporate multiple reviewer recommendations and comments to make decisions. Peer review is a vital part of academic knowledge generation. Reviewer contributions in a kind, complete, and constructive fashion provide an invaluable service.