The world’s largest multidisciplinary survey on research integrity is in danger of falling short of its goals after two-thirds of invited institutions declined to collaborate, citing the sensitivity of the subject and fearing negative publicity. That left researchers leading the Dutch National Survey on Research Integrity on their own to scrape many email addresses and solicit responses. The survey will close on 7 December, but the team has gathered responses from less than 15% of 40,000 targeted participants.
“It was supposed to be a collaborative effort, but it ended up as a satellite on its own in the Solar System, trying to send out signals,” says Gowri Gopalakrishna, a postdoctoral researcher at the Amsterdam University Medical Center (AUMC) who is coordinating the €800,000 survey.
Lex Bouter, who studies research methods and integrity at the Free University of Amsterdam (VU), began to plan the survey in 2016 to address a lack of data about questionable research practices and scientific misconduct. He wanted to ask all working academics in the Netherlands not just about how they conduct their research, but also about work habits, pressures, and other aspects of academic life. Bouter, a former VU president himself, assured the heads of other universities that the survey would not generate an institutional ranking of misbehavior.
But at a meeting in December 2019, some university presidents argued that a survey would just not be suitable for such a sensitive topic, Bouter recalls. Others found the survey too focused on bad behaviors, such as data falsification or cherry-picking of results. “I thought it was biased,” says Henk Kummeling, president of Utrecht University, which declined to participate. “If you only ask for questionable research practices, you already know what you will get out of the survey.”
Eventually, five out of the Netherlands’s 15 universities agreed to collaborate, on the condition that they could have a say on the survey’s setup and content. The questionnaire was broadened to include more questions about desirable scientific practices such as data sharing and open science. After the changes, the presidents of the five participating universities tried unsuccessfully to persuade the others to participate.
In a statement sent to Science, Leiden University President Carel Stolker says his institution declined to participate because the survey is methodologically weak, but he did not provide details. Kummeling says his university declined because he “did not get the impression the results would be useful for future policy” but denies that it was because of the sensitivity of the topic. “Everything related to integrity is sensitive,” he says. “But I have no problem at all with mapping out very clearly what is going on.”
But according to Gopalakrishna, the institutions did worry about bad publicity. “We tried to make the survey more balanced, but we did not manage to shake off the negative image,” she says. Jeroen de Ridder, a philosopher of science at VU who is not involved in the study, says he is disappointed that a unique opportunity to study research integrity across disciplines may be lost. He denies the survey has methodological flaws: “This has become the most careful and thorough survey one could wish for,” de Ridder says.
The participating universities helped Gopalakrishna by providing staff email addresses and sending reminders to fill out the questionnaire. To get responses from researchers at the 10 other institutions, Gopalakrishna and her colleagues had to scrape email addresses and send out the surveys unannounced. “This resulted in many more bounced and unopened emails,” she says.
Even some researchers at the collaborating institutions, wary of phishing scams, didn’t open their emails, says AUMC epidemiologist Gerben ter Riet, a co-investigator on the survey. “Researchers are flooded with emails and flooded with surveys,” he says. “It’s extremely difficult to stand out with something legitimate.”
As of 24 November, 13.3% of researchers affiliated with collaborating institutions had responded to the request, versus 9.5% of those at noncollaborating institutions.
After 7 December, the team will have to determine how representative the sample is. Bouter still expects it will be of value. “I am kind of disappointed about the proceedings, but with over 5000 completed surveys this still is the largest survey so far.”