Speaker of the House of Representatives Paul Ryan (R–WI, at lectern) and Senator Patty Murray (D–WA, second from right) thanked members of the commission on 7 September for their report.

Michele Freda, Commission on Evidence-Based Policymaking

What’s the evidence? Congress struggles to understand new report on evidence-based policy

Every member of Congress probably wants to use evidence to make better policy. But a hearing this week suggests it might be a mistake to press lawmakers for details, or ask what they mean by evidence.

Four members of the bipartisan Commission on Evidence-Based Policymaking gamely came to Capitol Hill in Washington, D.C., Tuesday to discuss their new report on how researchers could better use government records holding data on millions of Americans without violating privacy. The lawmakers who serve on the Oversight and Government Reform Committee for the U.S. House of Representatives were cordial to the members of the commission, created 18 months ago thanks to the efforts of Speaker of the House Paul Ryan (R–WI) and Senator Patty Murray (D–WA), for whom evidence-building capacity is a priority.

But those in attendance seemed to have a tenuous grasp of the commission’s focus—using administrative records to study the effects of hundreds of government programs. As a result, confusion permeated the 2-hour hearing, and commission members were forced to duck several questions that fell outside the purview of the study.

Not our job

Many Republicans, for example, were keen to hear how agencies could use the data to root out individual cases of waste, fraud, and abuse. Commission members said that wouldn’t really be possible, because the idea is to use the data solely for statistical purposes—and because all the records would be deidentified before researchers saw them. Several Democrats, meanwhile, hoped the commission’s report would bolster their arguments that President Donald Trump's administration has ignored scientific evidence in charting a course on climate change, immigration, and health care. Sorry, commission members explained: The evidence they were talking about comes from government records, not from a scientific study or a report by an expert panel.

Several lawmakers asked for the commission’s advice in thwarting cyberthieves. “Our report isn’t about cybersecurity, so we haven’t looked into that,” answered commission member Latanya Sweeney, director of the Data Privacy Lab at Harvard University.

Sweeney, the former chief technology officer for the Federal Trade Commission, is quite familiar with the problem of cybersecurity. But while serving on the panel she focused on another aspect of protecting digital data: namely, the technologies needed to prevent privacy breaches after researchers are given access to the data. And those researchers are not rogue players, emphasized commission chairperson Katharine Abraham, a former director of the U.S. Bureau of Labor Statistics. “They have already been vetted and given permission to use the data,” she explained after the hearing.

One lawmaker wasn’t so sure that those researchers can be trusted to be impartial. “Even though they are supposed to be bound by rigor, the vast sea of academics comes out left,” asserted Representative Glenn Grothman (R–WI). “And I think there is a bias toward trying to improve these programs, rather than getting rid of these programs, which is a lot harder. … My concern is that, if you talk to the right people, you can get the conclusion that you want.”

Abraham, now an economics professor at the University of Maryland in College Park, didn’t flinch. “Our report emphasizes the importance of rigor in the evaluation,” she replied. “And there are scientific standards for determining whether the conclusion of a study is valid or not valid. … As academics, what we can tell policymakers is that, if you do A, the outcome appears to be B. That still is not going to tell you what to do. … We’re simply advocating for evidence to have a seat at the table.”

Counting on census

Representative Carolyn Maloney (D–NY) may have come closest to linking the commission’s work to a current policy debate. Maloney asked about Republican attempts to make voluntary the American Community Survey (ACS), an annual questionnaire sent to 3 million households by the Census Bureau. The commissioners, chosen by both Republicans and Democrats, were united in their opposition to the idea. 

“How important is the American Community Survey for evidence-based policymaking?” Maloney asked. “Extremely important,” answered commission co-chairperson Ron Haskins, a senior fellow at the Brookings Institution in Washington, D.C.

“Some of my colleagues have suggested making it voluntary,” Maloney continued, noting that Canada’s decision to make its 2011 census voluntary resulted in much lower participation rates. “Would that affect the volume and integrity of the data?”

“A voluntary survey would ruin the ACS,” Haskins replied, explaining that a random sample allows researchers to accurately estimate the behavior of the overall population. “The decline in participation rates is not the biggest problem,” he said. “If the sample is no longer random, then none of the numbers are any good.”

The oversight committee has begun drafting a bill to implement some of the commission’s recommendations, and the timeliness of the hearing—less than 3 weeks after the commission issued its report—suggests the legislation will be on a fast track. Ryan and Murray have talked about a two-step process, with the first bill focused on the infrastructure needed to improve evidence-building. The idea is to show Congress that greater privacy protection and improved access can coexist before it tackles stickier issues like the commission’s proposal to loosen restrictions on the use of current federal data sets and to create a National Secure Data Service to oversee operations.

“The goal is to be able to protect the data. So along with having an open government, we have to maintain our freedoms,” said Representative Steve Russell (R–OK), speaking after the hearing that he led in the absence of the committee’s chairperson, Representative Trey Gowdy (R–SC). “The 22 recommendations from the report is a good framework to start.”

During the hearing, Russell questioned whether the Census Bureau is the best home for the new data service and whether such an entity is even needed. “I’m never reassured when I hear that we need to create a new agency to solve an existing problem at that agency,” Russell said after the hearing. “But I certainly respect the views of those who have studied the issue. And I think they made a good case for saying that, of all the agencies out there, the Census Bureau, with all its warts and flaws, is probably best equipped to at least start the process.”