iT@c/Flickr (CC BY-ND 2.0)

NSF reiterates policy on teaching good research habits despite its limitations

The National Science Foundation (NSF) in Arlington, Virginia, has decided to double down on its implementation of a congressionally mandated policy aimed at reducing research misconduct among NSF-funded scientists, despite a new report that notes problems with the agency’s approach.

In 2007, Congress approved a measure, the America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science (COMPETES) Act, that requires every university applying for NSF funding to certify that its students are receiving “appropriate” training in the responsible conduct of research (RCR). Although NSF gave universities great leeway to decide how to provide that training, its 2010 directive also suggested that schools conduct a “risk assessment” to determine who should be trained and what training they should receive.

In 2013 NSF’s Office of Inspector General (OIG), an internal independent watchdog, decided to see how well universities were complying with the requirement. And its new report, based on a survey of 53 institutions, identifies several areas of concern.

For starters, 23% of the schools with NSF research funds had no RCR training plan in place when OIG first contacted them. In addition, OIG found that none of the institutions surveyed had followed NSF’s advice to do a risk assessment.

The quality of the RCR training offered at universities is also suspect, the report says. Almost two-thirds of institutions rely solely on a short online program, it notes, even though most of the students OIG contacted said they would welcome a chance for meaningful interaction with faculty members on these thorny issues.

There’s also a problem with timing: Half the institutions said they don’t require students to complete RCR training before participating in an NSF-funded research project. “That obviously undermines the goals we are trying to achieve,” asserted Allison Lerner, NSF’s inspector general, in a presentation Tuesday to the agency’s oversight body, the National Science Board.

The OIG report also chides NSF for failing to comply with a congressional directive “to promptly develop and provide written guidelines and/or templates for universities to follow” to ensure compliance. “NSF’s awardees could benefit” from such guidance, it states.

Prodding the community

Notwithstanding those unresolved issues, NSF today posted what it calls an “important notice” reminding the academic community about its RCR obligations under the law. Previewing the notice for the science board, NSF Director France Cordova reminded members that Congress gave NSF “maximum flexibility in determining the full range of activities that would constitute appropriate [RCR] training.”

The notice (no. 140, dated 17 August) reminds universities that they are in the driver’s seat. “It is the responsibility of each institution to determine both the focus and the delivery method for appropriate training,” it states. “The training should be effective and appropriately tailored to the specific needs and circumstances at each university.”

The notice cites both the OIG report and a recent report by the National Academies of Sciences, Engineering, and Medicine that highlights exemplary RCR practices by individual universities. And it ends by asserting that “I believe we can all do more to achieve and demonstrate the effectiveness of RCR training.”

Cordova told the board that she “loved” the 14-page report by the OIG, which investigates scientific misconduct as well as all manner of waste, fraud, and abuse involving NSF funds. "It is wonderful that the OIG is monitoring RCR," she told ScienceInsider. "It will serve as a wake-up call to awardess, and I hope my Notice reinforces that message."

How about faculty?

Even so, the notice doesn’t embrace any of the OIG report's suggestions on how NSF could improve its oversight of university RCR plans.One such change would be to broaden the scope of the training beyond the current requirement that applies only to undergraduates, graduate students, and postdoctoral researchers working on NSF-funded projects. “It’s not the worst idea in the world to have faculty take RCR,” Lerner told the science board.

To bolster her argument, Lerner noted that 96% of the 70 scientists found guilty of plagiarism since 2012 were principal investigators, not students. That statistic, she said, is in line with the fact that a majority of NSF’s plagiarism cases involve grant applications, which cannot be submitted by students.

That idea resonated with a few board members. “Clearly, we should have in our statement that training is for students and faculty,” said Sethuraman “Panch” Panchanathan, executive vice president for research at Arizona State University (ASU) in Tempe and head of the board’s Committee on Strategy. “The culture of the lab is determined by the faculty member, yet somehow we left out the faculty.”

But ASU doesn’t require that faculty receive RCR training, Panchanathan admits. Instead, it simply encourages them to participate, on the assumption that the training will eventually diffuse down to their students.

None of the board members spoke up for any of the other suggestions in the report. And several expressed concerns about expanding the scope of training or tightening up on enforcement. In particular, they questioned why NSF would want to take such steps, fretted that any new requirements would increase the already heavy burden on institutions to comply with federal rules, and wondered whether greater scrutiny might be an overreaction to a problem that they said affects only a tiny number of researchers.

“Why would we want [NSF] to be the point of the spear, when it’s the universities who are responsible for ensuring appropriate behavior?” asked Vinton Cerf, chief internet evangelist for Google. “Somebody else should take up the burden.”

NSF already has the authority to reject grants from institutions who fail to certify they have an RCR plan, noted G. P. “Bud” Peterson, president of the Georgia Institute of Technology (Georgia Tech) in Atlanta. (OIG found 45 of 48 eligible institutions had come up with a plan and a process for monitoring RCR training before the study was completed.) “But has NSF ever turned down a proposal for that reason?” he asked. “Does anybody at NSF even pay attention to [the RCR plans]?”

Stephen Mayo, a biochemist at the California Institute of Technology in Pasadena, warned that “there’s a cost to increasing compliance” and that expanding the RCR requirement could further strain the ability to institutions to meet federal obligations. But other board members suggested that the negative publicity from a single incident of misconduct could pose much bigger problems for both their institution and NSF. “It only takes one case” in the headlines “to grab the attention of legislators and the public,” Lerner agreed.

NSF’s gentle hand

NSF’s current approach pays great deference to universities. NSF doesn’t necessarily know what a university is doing, notes medical ethicist Elizabeth Heitman of the University of Texas Southwestern Medical Center in Dallas, who co-authored a recently published study of how research-intensive universities are implementing the RCR mandate from NSF. In comments submitted before NSF adopted its current policy in 2010, Heitman and a group of professional ethics educators urged NSF to be more proactive, by reviewing RCR plans and offering guidance on how they could be improved. But that plea fell on deaf ears, she says.

“Why did Congress mandate RCR training to begin with?” she asks rhetorically. “Because NSF hadn’t acted on its own.”

Heitman says that NSF chose to implement the 2007 law in a way “that would be as easy as possible for universities to implement.” And given her own recent research, she isn’t surprised by OIG’s findings. “Most universities have responded in a way that isn’t meeting what NSF knows to be good practice,” she says.

The Academies’ report says that senior university administrators must lead the way in making clear to students and faculty that research integrity is a priority at their institution. And Peterson says he has no problem sending that message.

Speaking after the meeting, Peterson said it would be easy for Georgia Tech to mandate RCR training for all faculty. “All it would take would be for me to tell [the human resources department] that it is a condition of employment,” he explained. “Every university has things that faculty have to do in order to work here.”

At the same time, Peterson said he’d be wary of taking such a step without extensive discussions within the Georgia Tech community. “I’d want to ask my folks about how big this issue is on our campus compared to other issues we face,” he said. And he thinks that the metrics are important.

“I’d be reluctant to look just at cases of misconduct,” he said. “I can guarantee you that there have been more than 70 cases of bad mentoring,” he quipped, referencing NSF’s 5-year data on plagiarism cases. “Both are examples of a failure of responsible conduct. So I’d want to spend time looking at the magnitude of the [RCR training] problem and whether we feel what we are doing now is adequate.”

Clarification, 8/18/2017, 11:32 a.m.: This item has been updated to include the fact that the Notice has been issued and cite some of its content.