How Interactive Peer Review Works

Over the last couple of decades, new technologies and calls for a better system have challenged traditional peer review at scientific journals. Plenty of journals still do things the old-fashioned way, more or less—but these days, several open-access journals are employing alternative procedures, aiming to make peer review more open and collaborative. While the success of these initiatives has varied, several of these journals have become major players.

Interactive peer review can offer advantages for researchers. Among the most important: turning a blind, mostly one-way process into a conversation with established researchers. "The primary benefit, particularly for young scholars, is getting their work not just out into circulation but out into active conversation with the people in the field," says Kathleen Fitzpatrick, a visiting professor at New York University in New York City who studies how networked communication technologies affect scholarship. But there are disadvantages, too: Some of these journals have no or low impact factors, for now at least, and more conservative scientists on review committees may not give you much credit for publishing there.

[At ACP,] designated referees—who may identify themselves or not, as they wish—interact online with the authors and other interested scientists.

Among the most successful journals experimenting with more open peer review, different ones have different ways of integrating an interactive discussion into the process. At Frontiers, an initiative that was launched in 2007 by two neuroscientists at the Swiss Federal Institute of Technology in Lausanne that now publishes more than 200 journals in chemistry, biology, and the health sciences, the review process starts in pretty much the traditional way. The editor in charge passes all manuscripts that are submitted free of serious errors (i.e., ethical issues or unacceptable writing) on to invited reviewers, who send back independent reports. That's when things get interesting: Regardless of how favorable the reviews are, authors take part in an online discussion on the Frontiers interactive review forum, read the reviews, and discuss them with reviewers (who are still anonymous and can also interact with each other).

The discussion goes on until the reviewers decide unanimously either to accept or reject the paper; articles are rejected only if the reviewers find them to contain mistakes that authors are unable or unwilling to fix. Reviewers are asked not to consider the scientific or social significance of the work.

If the paper is accepted, the reviewers are named in the published paper and invited to write a commentary to be published at the same time. "It all works together to create an incredibly constructive process," says Henry Markram, the co-founder and co-executive manager of Frontiers, titles that he shares with his wife Kamila. "It’s a very thorough review, very fair, and very collaborative." After the article is published, the scientific community is invited to discuss a paper by submitting their own comments, which may also be published.

At the Frontiers journals, there's a lot of other postpublication activity encouraged. The community of scholars and the general public are invited to evaluate the work's academic quality and social relevance; those opinions are fed into what Frontiers calls its tiering system.

The average time from submission to publication is about 3 months. The Frontiers journals published more than 5000 papers that way last year, with a 15% to 20% rejection rate. They recently announced an alliance with Nature Publishing Group.

Other journals have opened up the review process almost completely. These include Electronic Transactions on Artificial Intelligence (ETAI), which was launched in 1997 by Erik Sandewall, a now-retired computer scientist from Linköping University in Sweden. Although it's now defunct, ETAI is widely regarded as a successful experiment. More than a dozen journals run by the European Geosciences Union and Copernicus Publications (including Atmospheric Chemistry and Physics [ACP], which was the first to be published in 2001), work on a similar model.

After passing a rudimentary preliminary review, where the editor in charge checks compliance with the scope of the journal and some basic scientific requirements, papers submitted to ACP are posted (and permanently archived) on the journal's online discussion forum, Atmospheric Chemistry and Physics Discussions. Over the following 8 weeks, designated referees—who may identify themselves or not, as they wish—interact online with the authors and other interested scientists (who are required to identify themselves). After the discussion, the authors have 4 weeks to publish a response to all comments and, if they do so satisfactorily, another 4 weeks to submit a revised manuscript. At that point, peer review reverts to a more traditional script, with the editor consulting the referees on whether to accept the paper for final publication or request further revisions.

As reported in July 2012 by ACP initiator and Chief Executive Editor Ulrich Pöschl of the Max Planck Institute for Chemistry in Mainz, Germany, ACP publishes about 800 papers a year and has a rejection rate of about 15%. Each paper gets an average of four comments, with one in five papers receiving a comment from the broader scientific community. ACP rapidly reached an impact factor of above 5, which has placed it among the highest-impact journals in atmospheric sciences, geosciences, and environmental sciences.

We'll have more on the pluses and minuses of the new peer review in the coming days. Stay tuned.

Search Jobs

Enter keywords, locations or job types to start searching for your new science career.

Top articles in Careers

A 3D plot from a model of the Ebola risk faced at different West African regions over time.
Dancing sneakers on pavement
siderailarticle x promo

Follow Science Careers