More than 150 prominent scientists and 75 scientific groups from around the world today took a stand against using impact factors, a measure of how often a journal is cited, to gauge the quality of an individual's work. They say researchers should be judged by the content of their papers, not where the studies are published.
Journal impact factors, calculated by the company Thomson Reuters, were first developed in the 1950s to help libraries decide which journals to order. Yet, impact factors are now widely used to assess the performance of individuals and research institutions. The metric "has become an obsession" that "warp[s] the way that research is conducted, reported, and funded," said a group of scientists organized by the American Society for Cell Biology (ASCB) in a press release. Particularly in China and India, they say, postdocs think that they should try to publish their work in only journals with high impact factors.
The problem, the scientists say, is that the impact factor is flawed. For example, it doesn't distinguish primary research from reviews; it can be skewed by a few highly cited papers; and it dissuades journals from publishing papers in fields such as ecology that are cited less often than, say, biomedical studies.
In what they've dubbed the San Francisco Declaration on Research Assessment (DORA)—a document drafted last December at the annual ASCB meeting and posted online today—the scientists write: "It is … imperative that scientific output is measured accurately and evaluated wisely." Their 18 recommendations urge the research community to "eliminate" the use of journal impact factors in funding, hiring, and promotion decisions.
Signatories include Science Editor-in-Chief Bruce Alberts (see his editorial); AAAS, Science's publisher; dozens of other editors, journals, and societies; as well as the Howard Hughes Medical Institute and the Wellcome Trust, which are major research charities.
"I see this as an insurrection. We don't want to be at the mercy of this anymore," says ASCB Executive Director Stefano Bertuzzi. He adds that the scientists aren't criticizing Thomson Reuters. "We're not attacking them in any way," he says. Instead, the resolution puts the blame on the research community for "the misuse of impact factors."
Bertuzzi says that his group realizes they won't change things overnight: "I see this as the beginning of a conversation." Still, he says, there are already signs of change.
For example, National Cancer Institute Director Harold Varmus is planning a pilot test that will ask researchers submitting biosketches with their grant proposals to describe their most important work instead of simply listing their key papers. Varmus said recently that he wants researchers to stop thinking that they must publish in only "certain hyper-prestigious journals." (In a similar move, the National Science Foundation recently changed its biosketch guidelines to emphasize "products" such as data sets, not just papers.)
Thomson Reuters did not respond to a request for comment.
*Update, 11:05 a.m., 17 May:
Thomson Reuters responded to the DORA in this statement, agreeing that: "No one metric can fully capture the complex contributions scholars make to their disciplines, and many forms of scholarly achievement should be considered." The company notes that the impact factor "is singled-out in the Declaration not for how it is calculated, but for how it is used."