Facebook knows your friends. Google knows your Internet searches. Your cellphone provider knows who you talk to. Yet, in the competitive world of scientific research, secrecy rules. Scientists tend to be hesitant to share too much about their work prior to publication, and the venerated peer-review processes at most journals are anonymous and opaque.
I can see that maybe some people don’t like to show potential weaknesses of their work, but I strongly object to this kind of thinking.
Some researchers, though, are pushing back against these norms, believing that openness can promote constructive communication and enable science to move forward faster. While some scientists remain concerned that sharing work via online “open notebooks” or preprint servers and publishing in journals with new peer-review models is risky (and perhaps introduces even more vulnerability into an already challenging career path), the researchers who are already participating insist that the benefits can outweigh the disadvantages—as long as you’re careful.
An open book
University of Toronto mathematician Dror Bar-Natan shares even the contents of his office chalkboard. A ceiling-mounted webcam posts images, including work labeled “A spherical loaf of bread,” “Taylor expansions,” and “Ants along a hoop,” to a publically accessible website he calls his “Academic Pensieve,” after the magical memory-holding device from the Harry Potter series.
Bar-Natan is one of a small number of so-called “open notebook scientists” who are pushing the boundaries of technology and data sharing by posting their work online, almost in real time. Conventional wisdom says that this type of sharing could be dangerous career-wise because it leaves researchers vulnerable to being scooped or criticized for posting something incorrect or foolish. Bar-Natan started as a professor in 1991 and didn’t create his pensieve, which also contains vast quantities of notes and other work products, until 17 years later, in 2008, when he already had tenure. He doesn’t worry about being scooped or criticized. “The fact is that I’m an exhibitionist,” he says. “I basically like showing my stuff.”
Tobias Osborne, on the other hand, started his online notebook when he started his first independent position in 2009. “I was definitely initially worried about getting scooped,” says Osborne, who is now a professor at the Leibniz University of Hannover in Germany. “However, I quickly discovered that that’s almost never going to happen.” That’s because it’s hard work to read and understand someone’s online notebook well enough to claim the work as your own, he says. Moreover, a post to an online notebook has a date stamp that at least informally indicates when the actual author had the idea; he thinks that reduces the temptation to try to scoop. Still, when he discusses his experience with colleagues, he says, “they say, ‘no, I could never make a blog; I’m so worried people would take my work.’ … I think the perception is that it’s a very likely thing, but I don’t think it’s at all likely.”
While researchers may fear that they will be scooped if they share, it’s important to recognize that not posting also carries risks, says Carl Boettiger, a postdoc at the University of California, Santa Cruz. Boettiger has kept an open notebook since his third year of graduate school, in 2010, and will start a faculty position in July. “Being closed has just as many risks as being completely open. … If you clamp down and release nothing, then you risk perhaps fewer people being aware of your work.” As evidence, he points to his role as a cofounder of rOpenSci, a group that creates software packages to support open science and which grew out of connections he made online, including via his online notebook. Another benefit of sharing work, he says, is that it allows others to point out mistakes so you can correct them. It can also help keep work organized, perhaps making it easier to submit to journals. Online resources may become increasingly important as journals and funders require more and more sharing of data, methods, and analysis, Boettiger says.
Despite these possible benefits, Boettiger doesn’t argue that his is a one-size-fits-all solution. “You have to find what works for you. It should be an intentional choice.”
Of course, sharing work prior to publication does carry real risks. You might share a result that helps a competitor make a major discovery or solve a big problem that you are both working on, which in the worst case could jeopardize your career. The added risk is likely minimal in disciplines where sharing is the norm, but if you’re the only one in your field who shares—or one of just a few—you could end up regretting it.
Even so, “I would make a plea to anyone contemplating making their work open to just give it a go and see what happens,” Osborne suggests. “Hedge your bets. Spend some time doing open work and some doing closed work and compare the results.”
Osborne notes one more advantage. “The main hurdle that young scientists face is getting their work noticed,” he says. “There is a huge first-mover advantage to be had at this moment in time. This is a good opportunity for people who want to experiment with open notebook science and sharing stuff online. Now is the moment to get that first-mover advantage, because the space is empty.”
For those who aren’t quite ready to put all their work online, preprint servers can offer some of the same benefits. Researchers who use preprint servers—arXiv is the most well-established one, but there are others—usually post manuscripts there at about the same time that they submit them to a peer-reviewed journal. Until the work is peer reviewed, it is up to readers to evaluate it for themselves. “It’s just a very convenient and fast way of disseminating information,” says Michael Lässig, a physicist at the University of Cologne in Germany. “It’s a very democratic means of sharing.”
When arXiv was founded, it was viewed as a way to balance the desire for rapid communication of scientific results with the pressure to claim priority. “Once something appeared in arXiv, other people would have to cite it, so it reduced the risk of being scooped during the delay of review,” Lässig explains. “It’s sort of a self-reinforcing process: If everybody uses the arXiv, then the arXiv is also the universal way of securing the intellectual property.” Moreover, if editors and reviewers for peer-reviewed journals notice similarities between a submitted manuscript and work already posted on arXiv, they can point the authors to the preprint and ask them to discuss and cite it. “That’s a way of creating just distribution of intellectual credit,” Lässig says.
In addition to establishing priority, the rapid dissemination offered by preprint servers can help accelerate progress, says Richard Sever, who in 2013 cofounded the biology-focused preprint server bioRxiv. “People talk about how much scientific time is lost waiting for papers to come out. If it takes a year for something to go through peer review, that’s a year that people could have been working on that subject. … People want to get their work out quickly and make it available to the world, and there are altruistic reasons for doing so, and there are reasons of self-interest for doing so. The good thing is that hopefully those two things align and science benefits.” Lässig cites as an example a preprint he and his co-authors posted on bioRxiv last November about the ongoing Ebola outbreak, which could have immediate implications for the efforts to combat the disease. Now the information is publicly accessible for those who need it, and Lässig gets credit for the work.
Sharing can also lead to productive exchanges between authors and readers. “When you put something out, you often get responses from colleagues right away,” Lässig says. Just as in the open notebook case, these discussions can help authors fix mistakes before they become an indelible part of the scientific record. “Maybe someone points out an error and you get a chance to correct it. If you don’t show your paper to people, you don’t get that chance.”
Despite these benefits, the idea has not fully caught on with biomedical researchers. At least part of the reason is community norms, says Ohio State University physicist Ralf Bundschuh, who is chair of the arXiv Scientific Advisory Board. “Unless everybody thinks of a preprint archive as the medium to establish priority, there is still a risk of being scooped,” he says. If there’s no widespread agreement and no standard, another scientist might be able to claim priority after publishing a later, peer-reviewed article that is similar or related to a manuscript previously posted in a preprint server. It is up to the grant reviewers and tenure and hiring committees in each field to determine how they value different forms of dissemination. Moreover, in fields where sharing is the exception rather than the rule, posting results on a preprint server could have the same negative repercussions as sharing work in an open notebook: It could give competitors a leg up.
Another barrier is unclear journal policies: Does posting on a preprint server count as prior publication and therefore disqualify your manuscript from consideration by peer-reviewed journals? Many journals have clarified or changed their policies to make it clear whether they will consider manuscripts that have been posted on preprint servers. Nonetheless, some publishers and journals, including the American Society for Microbiology and some journals of the American Chemical Society, will not publish work that has appeared on preprint servers. So authors should make sure they are familiar with the policies of the journals in their field before deciding how to proceed. There’s even a Wikipedia page listing journals by preprint policy to help authors navigate this landscape.
Some journal policies can still leave some wiggle room. Consider Science’s policy, for example (emphasis added): “Distribution on the Internet may be considered prior publication and may compromise the originality of the paper as a submission to Science, although we do allow posting of research papers on not-for-profit preprint servers such as arixv.org.” That also includes bioRxiv, says Deputy Editor Andrew Sugden. Beyond those two platforms, though, “it’s difficult to articulate general guidelines, because venues for posting pre-prints are currently proliferating and evolving so rapidly,” Deputy Editor Jake Yeston writes in an e-mail. “In the case of other forums, it’s best for authors to contact editors directly beforehand, so we can evaluate the implications carefully.”
Lifting the peer-review veil
Peer-reviewed publications remain the gold standard for research output. Nikolaus Grigorieff, a lab head at the Howard Hughes Medical Institute’s Janelia Research Campus in Ashburn, Virginia, takes it upon himself to introduce transparency into the peer-review process for the papers he publishes. His lab website includes a page called “The Paper Rejection Repository,” where he posts rejection letters for papers that ultimately were published elsewhere. “There’s no shame in showing that some things haven’t worked out,” he says. “I can see that maybe some people don’t like to show potential weaknesses of their work, but I strongly object to this kind of thinking.”
Grigorieff’s practice may not be the next big thing, but some journals are experimenting with other ways of lifting the veil of secrecy that traditionally hangs over peer review. Some, for example, publish reviews and decision letters along with published papers; some even include the reviewer’s name. These moves toward increased transparency affect all participants in peer review: authors, reviewers, and readers.
In January 2014, marine ecologist Nicholas Dulvy of Simon Fraser University in Canada had a paper published in eLife, a journal that publishes the decision letter alongside each published paper. At first, Dulvy says, “I was apprehensive, but I couldn’t think why. It was just something new, and we decided to embrace it. … I would encourage authors to embrace open peer review. I think it generates a much more constructive tone rather than an adversarial one, and I think that that’s really good for science.”
It’s worth noting, however, that few of the journals experimenting with open peer review are among the leaders when it comes to impact factor or reputation, at least for now. And for better or worse, journal prestige and impact factor are still important criteria when evaluating early-career scientists. Choosing a journal just because it has open peer review may not be a smart career move if you think you could publish the work in a higher-profile journal instead.
The situation can be even more complicated for reviewers. Reviews often contain valuable critiques that can help readers evaluate the work, so open reviews may be good for science. Furthermore, relinquishing anonymity allows reviewers to be recognized for their contribution as reviewers, which may be especially valuable for researchers early in their careers, who have not yet filled out their CVs with peer-reviewed publications. Add these two points together, and open reviewing starts to look like yet another venue where early-career scientists can demonstrate their knowledge and insight (or embarrass themselves if they get it wrong).
“There are papers already out there that I’ve reviewed that I kind of wish my reviews were published with because I think it’s important for any reader of the paper to take into account some of the concerns that I and other reviewers were bringing up—and I think that’s true for a lot of science that’s out there,” says Michelle Wirth, an assistant professor of psychology at the University of Notre Dame in Indiana who has published a paper with F1000Research, another journal experimenting with open peer review. “It’s an interesting, different idea, and we’ll have to see what the pros and cons are over time.”
On the other side, reviewers have long relied on anonymity to be able to provide an honest critique of a manuscript without fear of retribution or being perceived as currying favor. Losing that anonymity could compromise the integrity of the process.
Journals will likely continue experimenting with peer-review models. Meanwhile, the solution for individual reviewers can be simple: Don’t agree to review a manuscript if you don’t feel comfortable. “In the end, the most important thing is that you write an honest review,” says Grigorieff, who has acted as a nonanonymous reviewer for eLife. “If having your name next to it might influence you or prevent you from doing this, it’s probably better not to have your name next to it.” That’s a point that can also be applied to the other open ways of disseminating your research. “I think transparency in all walks of life is a good thing,” Grigorieff says, “but it sometimes comes with drawbacks, and one has to balance these two.”