A journal published a paper today that reveals a set of folding constraints in the design of RNA molecules. So far, so normal.
Most of the data for the study come from an online game that crowdsources solutions from thousands of nonexpert players—unusual but not unique.
But the lead authors of the paper are the players themselves. Now that is a first. And there's a twist: The journal nearly delayed publication because of "ethical" concerns about authors using only their game names.
The game is called Eterna, and it made a big splash in 2014 with a paper in the Proceedings of the National Academy of Sciences that had 37,000 players as co-authors. The goal was to see whether nonexpert humans can do better than computer algorithms at designing RNA sequences that fold into particular shapes. And indeed, the humans won, even after the computer algorithms were endowed with insights from the human folders. When it comes to the biophysics of RNA folding, John Henry still beats the machine.
That 2014 study was led by card-carrying scientists Adrien Treuille and Rhiju Das, biophysicists at Carnegie Mellon University in Pittsburgh, Pennsylvania, and Stanford University in Palo Alto, California, respectively. The two researchers created the game in 2009. (They both cut their teeth on scientific game design as postdocs in the lab of David Baker at the University of Washington, Seattle, where the blockbuster game FoldIt was conceived.) Since then they have massively scaled up the process and hooked the game to a real-world automated lab that actually tests the folding predictions made by players against the 3D structure of the RNA molecules. They call it the Eterna Massive Open Laboratory.
Human versus machine
Today's paper shows how far the effort has come. Among the game's thousands of RNA design "puzzles," there seem to be a small set that are particularly difficult. Among the most challenging structural features to figure out is symmetry, where an RNA strand folds into two or more identically shaped loops. The Eterna game includes an interface for players to propose hypotheses about how particular RNA structures will or will not fold into particular shapes. Those were distilled into a set of "designability" rules. The question was: Do only human designers struggle with thorny design problems, or do computer simulations tussle too?
The answer is that the computers struggled just as much as the people. Researchers report that three of the best existing computer algorithms, running on a supercomputer at Stanford, struggled to solve the very same RNA design problems as the humans. The result shows that the human "designability" rules do indeed correspond to problems that are hard not just for human brains but also for computers, the team reports today in the Journal of Molecular Biology. In fact, the hardest puzzles that could be solved by experienced Eterna players were unsolvable by the computer even after days of crunching. And to help improve the algorithms, computer scientists now have a set of benchmarks—the Eterna100—to gauge the design difficulty of RNA structures.
"This paper is a significant contribution," says Jane Richardson, a biophysicist at Duke University in Durham, North Carolina. "The new design principles seem very understandable now that they've been described, and attention to them should indeed help automated design." She notes that the game itself has become a useful tool for scientists. "One of our lab's former students is an avid Eterna player who has also gotten insights from it for his own current research projects."
But publishing a paper led by game players posed a conundrum: Some of the contributing authors were listed only by their game names rather than real-world names. "A generally accepted criterion for authorship is that authors must take responsibility for the integrity of the paper," says the journal's editor, Peter Wright, a biophysicist at the Scripps Research Institute in San Diego, California. "In my view, identification of an author only by a screen name, which can be changed at an instant, is inappropriate, since that person could not be contacted if issues of accountability arose."
In the end, the players who are listed as co-authors simply agreed to have their full names listed. "It's really a bit amusing," says lead author Jeffrey Anderson-Lee, a computer systems manager at the University of California, Berkeley, and an avid Eterna player. "I guess it's stretching [the journal’s] boundaries a bit."
What's next for the citizen scientists? Many more papers, Das says. "Over the last 2 years, Eterna has offered players the ability to define their own microprojects, and they’ve carried out about 80 so far. If the community can distill these insights into concise manuscripts there should be a steady stream of player-written papers coming out."