AMSTERDAM—Is experimenting with e-cigarettes among young people associated with a higher risk of smoking tobacco, as an influential study published 2 years ago in The Journal of the American Medical Association (JAMA) found? Can exposure to a natural environment help you recover from stress, as researchers claimed in a landmark 1991 paper? And is the “10 commandment effect”—the phenomenon that reminding people of moral standards makes them less likely to deceive —real?
We may soon have fresh answers to those questions, thanks to the first research fund specifically dedicated to replication studies. The Dutch Organization for Scientific Research (NWO) today announced the scheme’s first nine grantees ; all of them plan to replicate a study that had a major impact in their field but also raised questions—or eyebrows—for some reason. NWO has made €3 million available for the pilot program, to be awarded in three annual rounds.
Two types of studies were eligible for funding: reproduction research, essentially a reanalysis of the data collected in a previous study; and replication, which involves the collection of fresh data using the research protocol for the original study. Applicants had to show that the study they wanted to replicate had “cornerstone” status in the field, and that the replication was relevant, with a special focus on impact on health policy and medical guidelines.
The idea came from cognitive psychologist Daniël Lakens at Eindhoven University of Technology, who argued in a 2012 paper that stimulating replication studies would improve psychological science, and later challenged NWO to foot the bill for a replication program in the Netherlands. Lakens’s proposal was targeted at psychology but he says he’s happy that about half of the grants are in medicine. “I hope something will happen with the results,” he says. “Guidelines might have to be changed, or in case of a confirmation, some doctors will be able to sleep a little better.” The large number of applications, 85, shows there is a need for this type of studies, Lakens adds: “I hope more countries will soon follow.”
Among the grantees is Evert de Jonge of Leiden University Medical Center, who will redo a randomized controlled trial published in JAMA 2016 that showed that keeping oxygen pressure low in the veins of intensive care patients reduced mortality. Nynke van der Laan of University Medical Centre Utrecht will replicate a study published in Science in 2009 that provided the first explanation for the neural basis of successful self-control in humans.
The oldest study up for replication is one published in Science in 1960 by Eckhard Hess and James Polt of the University of Chicago in Illinois. They showed that people’s pupil size increases when they’re watching “emotionally toned or interesting images” such as pictures of babies or naked people. The study has been cited 625 times and is still regarded as one of the most important ones in the field of human-machine interactions. But doubts have always swirled around the study, says Joost de Winter of Delft University, who will replicate it. Other factors, such as the brightness, contrast, and colors in the pictures may have determined the subjects’ pupil size, De Winter says, and those could be related to the attractiveness of the images.
De Winter plans to replicate the study using the exact same materials used by Hess and Polt—now stored at the Drs. Nicholas and Dorothy Cummings Center for the History of Psychology at the University of Akron in Ohio—but will also repeat the experiment using computer screens and eye trackers, to check whether the results depend on the technology. “It will be more than just a replication,” De Winter says. “It will be extremely interesting to have a good look at the original data and figure out how exactly the original authors worked.”