LONDON—The anticipation is over. Scientists and higher education officials in the United Kingdom are now poring over the findings of an influential evaluation of university research released here today. The periodic and typically controversial report, now called the Research Excellence Framework (REF), highlights overall improvement in research quality across the United Kingdom. The government funding councils that ran the massive review evaluated individual research departments, but didn't rank them or the universities. As usual, though, others quickly crunched the numbers and officials started bragging.
University College London (UCL) was first to trumpet its ranking, claiming the top slot that had in the previous evaluation gone to the Institute of Cancer Research and before that to the University of Cambridge. Cardiff University boasted of its "meteoric rise" in the rankings. Meanwhile, the Russell Group, a consortium of UCL, Cardiff, and 22 other large universities, said the REF’s results justified concentrating limited funds into the best performing institutions. “The volume of world-leading research in Russell Group universities is more than double that in other universities," said Wendy Piatt, the group’s director general in a statement.
Universities care not just about the prestige, but also about the money involved. The Higher Education Funding Council for England (HEFCE) and three similar councils provide unrestricted grants to universities. The individual department scores, which for the first time include a measure of societal impact, will influence the annual distribution of the £2 billion in government funding. “The REF2014 is hugely significant, more so than any league table or university ranking," said Michael Arthur, UCL's president and provost, in a statement. But the universities won’t learn exactly how the United Kingdom will use the information to parcel out funds until early next year.
The detailed peer review of 154 universities in the United Kingdom shows overall improvements in research since the previous assessment in 2008. Twenty-two percent of submitted research was given four stars, the highest rank and considered "world-leading," compared with 14% in 2008. Half of the research won three stars, rising from 37% in 2008. David Sweeney of HEFCE attributed the improvement to overall increased funding of research at universities.
Others have speculated that universities learned how to more intelligently pitch their research to the REF, for example by being more selective in the researchers whose work they submitted to the panels. "With the investment that universities are making in REF staff, you'd expect them to do a better job," says Sarah Main, director of the Campaign for Science and Engineering, a nonprofit in London that advocates for research. "It's a very competitive environment."
The evaluation is enormous. Over this year, more than 1100 experts evaluated the work of more than 52,000 academics, examining 191,150 papers or other research products. In addition, universities submitted nearly 7000 case studies demonstrating the societal impact of their research. Universities were also scored for their "research environment," including strategy, facilities, and professional development. "This is the largest research assessment exercise in the world," Sweeney noted with obvious pride.
Yet the process has been controversial. An initial plan to give major weight to citations, grant income, and other quantitative metrics was scaled back, delaying the assessment. Others have criticized the peer-review process as too cursory and vulnerable to the intrigues of academic politics. Janet Finch, a sociologist at the University of Manchester who chaired one of four supervisory panels, defended the REF as rigorous, fair, and transparent: "The level of confidence that I have in these results is very high," she said at the press conference.
Another complaint is the amount of time involved. "Given the monetary and human resource costs of the exercise, we simply cannot afford not to ask how well the whole system works," wrote Nicholas Stern, president of the British Academy and Paul Nurse, president of the Royal Society in Times Higher Education last week. HEFCE expects to release a report in March about the administrative burden that REF places on universities. It has also commissioned an independent review of its methods, which should be completed in the spring.