When she was a medical doctor, Kristen Grace specialized in fertility treatments. Alongside her medical practice, she carried out research in the lab and followed new developments in the field. In her current post as a health scientist administrator (also called a scientist-investigator) at the U.S. Office of Research Integrity (ORI), she does many of the same things: She continues to keep up with findings in the science literature, and she relies on insights borne of her experience to elucidate the problem at hand.
At the ORI, Grace works side-by-side with a handful of other former bench scientists, from various fields, who have traded direct involvement in research for one of science’s most important functions: monitoring and protecting the integrity of science and the scientific literature.
“An assumption that a manipulated image necessarily equals research misconduct is incorrect, since inconsistencies may be due to other factors, including honest error.” —Shara Kabak
From surgeon to sleuth
The move from the OR to the ORI may seem unusual, but Grace has often bucked convention. She waited until her son entered kindergarten to complete a bachelor’s degree, followed that up with a medical degree and, after that, earned a Ph.D. in developmental biology at the State University of New York, Stony Brook. After completing a residency at the Albert Einstein College of Medicine in New York, she left clinical work to pursue research at the National Institutes of Health (NIH). Her work there focused on sperm proteins that can trigger immune responses, leading to infertility.
Then, 4 years ago, as her time at the NIH drew to a close, Grace learned about the opportunity at the ORI. “It was a fortuitous opportunity for me because I wasn’t interested in practicing in the clinic,” she says.
Grace was drawn to the job because of its focus on ethics. While working in reproductive medicine, she had observed that the challenges faced by clinicians in her field are never simply clinical or diagnostic; they inevitably involve ethical decisions as well. Surrogate motherhood, frozen embryos, and egg donation are good examples: Each advance brings with it a tangled set of new dilemmas. “The ethics of science and what we do in science and how that affects society as a whole was something that was always in the background when dealing with reproductive research,” Grace says.
In research labs, she has noticed, ethical decisions are also made against a complex backdrop of real-world complications. When Grace and her colleagues examine misconduct allegations, background issues such as the pressures of funding and the difficulties of running a clinical trial must be weighed. In one recent case, a suburban Chicago cardiologist was found guilty of scientific misconduct for pressuring trainees to change or fake scores on a heart test so that patients would qualify and the clinical trial could go forward. “In this case, the PI [principal investigator] falsified patients’ echocardiograms to make patients appear sicker than they actually were so they could be enrolled in the study.” Because she had been on the researcher’s side, faced with the need to adhere to strict inclusion and exclusion criteria, Grace was able to “tease out details that somebody who hadn’t been on the floor doing research wouldn’t have been able to do,” she says.
What the job is like
When, recently, Grace examined a falsified figure published in a journal, her experiences as a physician and a scientist told her that the doctored image was likely to cause trouble. It could, she realized, jeopardize subsequent research and lead to inappropriate medical treatment. Meanwhile, her experience as a scientist-investigator told her that the alteration was probably just the tip of the iceberg. “Looking at that figure, and knowing the data and the way it has progressed throughout the years, I knew that this data had to have been falsified more extensively over a much longer period of time,” she says. It was, in other words, a serious problem for science.
The structures of science are built to tolerate honest mistakes, but scientists must have confidence that the literature accurately reports the results of actual experiments. That is why fabricating or changing a result is so shocking, says Susan Garfinkel, acting director of the Division of Investigative Oversight at ORI. “Scientists get really enraged when they think that there’s something wrong with the data,” she says. “When you publish a paper, the people reading it have to believe that it’s correct. If there’s any chance that it’s not correct, then you can’t depend on that to support your own research.”
Reports of research misconduct come from various sources. Sometimes, scientists notify Garfinkel’s office directly when they see questionable data in a journal article. Other cases are sparked by inquiries from journal editors, who noticed something suspicious in a submitted manuscript. No matter where the report comes from, the researcher’s institution—not the ORI—is responsible for carrying out an inquiry to see whether an allegation has merit. If they decide it does, a formal investigation ensues, carried out by the scientist’s institution.
During such an investigation, ORI investigators work with the institution’s research integrity officer (RIO) to guide proceedings and serve as expert consultants to:
- help with technical issues such as image analysis
- answer questions on how to comply with federal regulations
- offer advice on how to deal with obstructive defendants or aggressive attorneys
ORI investigators also acquire the data they need to review the case, including a forensic copy of the hard drives related to the allegations.
Once the institution has completed its investigation, ORI investigators examine the research records related to the allegations, including published papers, grant applications, laboratory notebooks, computer files, manuscripts, abstracts, theses, and presentations. ORI investigators usually work closely with institutional RIOs; sometimes, though, they work independently, using their own tools to repeat forensic procedures or analyze statistics.
When there’s a finding of research misconduct, the case is reviewed by all of the scientists in Garfinkel’s division, as well as by attorneys. The legal standard for a misconduct finding is “preponderance of the evidence,” which means investigators must conclude that misconduct is more likely than not. But because ORI findings are public, Garfinkel says, her office often uses a higher standard. “This includes evidence to prove the data are false, evidence to prove the [accused party] was responsible, and evidence to prove that the falsification is a significant departure from accepted practices,” she says. Investigators must also show that the misconduct had an effect on the research in that field.
The scientists currently working in the Division of Investigative Oversight have backgrounds in immunology, genetics, statistics, chemistry, and veterinary science. Without access to such broad knowledge and first-hand experience, the office would find it difficult to make a determination about many specific cases, Garfinkel says.
Garfinkel studied molecular biology as an undergraduate and received a doctorate in genetics from George Washington University. By the time she completed her postdoc, she had grown tired of lab work, so she took an administrative job. “I didn’t feel I was doing anything to contribute to the scientific community, or make a difference,” she says. She also missed the fast pace of lab work. When the opportunity to work at ORI arose in 2003, she took it. “Because every single case that you work on is a little bit different, you just don’t get bored,” she says.
ORI investigators receive no formal training beyond their scientific training—which, however, is essential. An understanding of what lab culture is like is also necessary. “People who have worked in a lab have unique insight into these dynamics,” Garfinkel says, but the scientific perspective is not exactly right for the job: Over time, investigators learn to think more like attorneys, she says. “There’s kind of a steep learning curve when you first come in, to change the way you think about things.” The researchers also must train themselves in forensic analysis techniques for both images and computers, and in how to perform statistical tests that can detect fabricated data.
Better detection methods
Today, scientists with no official investigative role are running their own data-forensics operations. Some even set up blogs with entries pointing to suspicious-looking figures or facts and claiming that they’re fraudulent. Shara Kabak, a scientist-investigator at ORI, says that while her office doesn’t discourage others from looking for wrongdoing, it encourages reporting allegations through appropriate channels: the university or ORI. Proper reporting of allegations, Kabak says, can reduce the chances that crucial evidence will be lost or destroyed. It can also reduce the risk of damaging a researcher’s reputation, she says. “An assumption that a manipulated image necessarily equals research misconduct is incorrect, since inconsistencies may be due to other factors, including honest error.”
Kabak was introduced to research misconduct while serving on an NIH committee on scientific conduct and ethics. After receiving a doctorate in immunology from the University of Chicago, she moved to Washington in 2007 to become a AAAS (publisher of Science Careers) Science and Technology Policy fellow, working at NIH and following investigations involving NIH intramural scientists. Following a brief stint as a science writer at the National Eye Institute, she accepted the scientist-investigator job at ORI.
ORI’s workload is growing. Last year, the number of cases reported to the office doubled. Kabak says that cases are also becoming more complicated, often involving more than one person. Fortunately, detection methods have also improved, allowing investigators to do a better job of tracking patterns of behavior. “Normally something might have come in and we would have found one thing. Now, with a little bit of digging, it’s easy to find more instances if there’s misconduct there,” she says.
The hub of a big wheel
Like many nontraditional science jobs, research integrity is a niche career. The center of that universe, the ORI, is small: Right now, in addition to Garfinkel (the acting director) the office employs six investigators, and three subject-matter experts who do essentially the same work. ORI recently advertised an open position for a scientist-investigator specializing in organic chemistry or molecular biology, with a specified annual salary range of $106,263 to $138,136; the ad closed a few days before this article went to press. (ORI is also hiring a new director following the departure of David Wright, who resigned in March while issuing a scathing critique of what he called the 'dysfunctional' bureaucracy at the Office of the Assistant Secretary for Health, as ScienceInsider reported.)
But the world of research integrity is larger, with investigators at every research institution. With caseloads increasing, that world is likely to get larger still. We’ll present more information about these university-based research-integrity jobs in a future Science Careers article.