You've just received an e-mail from a journal editor asking if you could review a manuscript. The research is directly relevant to your field, and you've got a month to do the review. You're flattered this editor knows and trusts you, but you're filled with doubts. You've never reviewed a manuscript before. What exactly will she expect from you? How do you go about doing a proper peer review?
Like many other academic job skills, reviewing skills are expected but rarely taught. Most people learn "by doing," says Dario Sambunjak, a senior editor at the Croatian Medical Journal ( CMJ) and research fellow in education and scientific method at Zagreb University School of Medicine in Croatia. Yet there is much you can do to prepare, and it's worth putting in the time, because reviewing manuscripts has many indirect benefits.
What's in it for you?
More than nine in 10 of the academics interviewed in an international study said they reviewed manuscripts to play their parts as members of the academic community. Two-thirds said they do it to pay back the benefit of having their own papers reviewed. A peer review is a gift to others, says Ana Marusic, joint editor-in-chief of CMJ and past president of the international Council of Science Editors. But this isn't to say there is nothing in it for the reviewer.
Reviewing manuscripts means applying your critical skills to research that's often cutting-edge. It also exposes you to new science. Either or both of these experiences can lead to new ideas and new approaches to your own work, Marusic says. Reviewing other people's manuscripts may also help you improve your own writing and basic science skills. "You can anticipate problems and deficiencies, and maybe you can solve some of them ... in your articles if you know already what reviewers will be looking for," Marusic says.
Editors have a database of reviewers their journals have built up over the years, but they often look for new, reliable reviewers. Editors may notice you from a literature search, a conference, or a recommendation from a colleague. "The moment you are a good author publishing papers, you might expect to get invitations for peer review," Marusic says.
Reading the scientific literature and participating in journal clubs are great ways to start preparing to review manuscripts. You'll develop your critical skills and build knowledge of your field and the different journals in it. Some journals, such as those on BioMed Central, now publish peer reviews and authors' responses alongside research articles. That kind of forum "can be a good place to ... see how good reviews are done," Sambunjak says.
Before you go in search of official reviewing gigs, review some colleagues' manuscripts; most scientists are happy for the opportunity to improve their manuscripts before they send them off to journals.
Once you feel ready to become an official reviewer, offer your services to smaller journals, as these usually have a more difficult time finding reviewers, Sambunjak says. One way to get started is to share reviewing duties with your supervisor--but make sure it's been cleared with the editor first. Sharing them with anyone without permission is "a violation of confidentiality," says Melissa Anderson, an ethics researcher at the University of Minnesota, Minneapolis.
What editors expect
When an editor asks you to review a manuscript, he or she is looking for a critical analysis of a paper's scientific rigor, significance, and relevance. "The reviewer is there to advise about the science and the originality and the interest" of the manuscripts editors receive, explains U.K.-based writing and publishing coach Elizabeth Wager. By providing such expertise, reviewers help journals decide what research to publish and, along the way, improve the manuscripts.
It's important to make sure your expertise is a good match for the manuscript you are asked to review. "If you aren't competent to review and judge the work [of the manuscript's authors], you shouldn't be doing it," says Anderson. Also make sure you'll be able to submit the review to the editor in the time she specified--usually a month or so. "If you don't make the journal's deadlines, it delays publication and could put the authors at a disadvantage," Anderson says.
Writing a review
There is no algorithm for reviewing a paper, but general guidance abounds (see box). Editors often send instructions and checklists together with the manuscript, and most journals provide information on their Web sites.
Experts recommend reading the whole manuscript once to get a general impression of how it reads and whether it makes sense. At this stage, you're looking for major flaws and missing information. Next, look at the paper in more detail. "You have to use your knowledge of the field and judge the scientific novelty and the basic design of the study," Sambunjak says. Highlight points that are not supported by the data, have alternative explanations, or could be presented better. Look to see if there are "any experiments that are not difficult to perform and would [reinforce] the strength and the argument provided in the paper," says Lubomir Tomaska, a genetics professor at Comenius University in Bratislava in the Slovak Republic, who teaches the publishing process to undergraduates.
When you write the review, separate your comments to the editor from instructions to the authors on how to improve it. A good review should offer some specific, obtainable, concrete, and opinionated comments, Sambunjak says. "Be thorough and constructive," Wager adds. Whether you found the article good or bad, you need to detail why, referring to how appropriate the study design or the methodology is, for example. Always support your points with sound evidence and arguments. It's also a good idea to write your comments down in a numbered list, clearly marking which changes are major and compulsory, or minor and optional. "That makes it easier for the author to respond and for the editor to affirm or reject your suggestions," Anderson says.
"The worst review would be one that misses the mark so that it's clear that the [reviewer] doesn't understand [the manuscript] and was dismissive of it," Anderson says. Reviewers who "humiliate or put people down or who dismiss a manuscript without offering any advice for improvement are not offering a good service."
Wager says you should expect a review to take you 4 to 5 hours, on average, depending on how long the paper is, how difficult it is to read, and whether you have to consult the literature about the researchers' study methods.
Ethics and other pitfalls
When you are a peer reviewer, "you are the first person seeing the most novel result in the field," Marusic says. "This great privilege comes with great responsibility." It may be tempting to "borrow" an idea from another group's work. But "you can't do anything with the information you get" from a manuscript under review, Anderson says. You need to wait until it appears in the published literature.
Watch out for potential conflicts of interest. You don't need to decline reviewing a manuscript just because the authors are former students or because they are in a friendly or competing lab. "But if you feel you can't give a fair review because you are in direct competition with that team, ... it's important that you reveal" the conflict to the editor and decide together whether to go ahead with the review, Anderson says. Some other biases are more insidious. Reviewers "really should avoid looking at the name of the author and the country where he or she comes from and then judge the quality of the research in that context," Tomaska says.
Early-career scientists may find themselves in a difficult position if they review the work of an important senior scientist for a journal that discloses the identity of reviewers. They may be worried, perhaps with cause, about harming their career prospects. "Then they may be very reluctant to be critical and unbiased in their review," Marusic says. Talk with your editor about any concern you have. Also be aware that, even in a closed review system, it may be difficult to guarantee your anonymity: Although rare, breaches of confidentiality do happen, or perhaps you're in a subspecialty that is so small that you'll be easily identified as the reviewer.
It's very difficult to detect fraud, and no journal will hold you responsible if you miss it, Anderson says. "You don't have the raw data. ... You can't tell whether the data have been fabricated," Wager says. If, however, you suspect that images have been manipulated, or you know that parts of the manuscript have been published somewhere else, you have the "responsibility to point out that to the editor," Anderson says. Also tell the editor if you feel the research itself is unethical because it doesn't have the proper safeguards for research in humans or animals, Wager adds. Finally, there's one more important skill you should develop as you get more established: learning to say "no." "I wouldn't advise young scientists to reject reviewing a paper. ... It's a good exercise," Tomaska says. But "you have to find a good compromise between ... helping the other peers and the research area and keeping your time free for important activities like doing research, doing teaching, and writing grants," he adds.