Despite declining revenues and criticism from U.S. President Donald Trump, U.S. media outlets—even small ones—are still able to sway public discussion. That’s the conclusion of a 5-year study, which found that when news organizations ran a number of stories on controversial topics like water quality and climate change in close succession, they significantly boosted public conversations about these topics—and even changed some people’s minds.
“I’m impressed by the sheer audacity of this study,” says Lynn Vavreck, a political scientist at the University of California, Los Angeles, who wasn’t involved in the research. If the media are able to boost public discussion, they can help the democratic process by increasing participation in important national decisions, she says.
Assessing the influence of news media is tricky. Researchers can’t peer into voting booths or people’s living rooms, and news organizations aren’t typically willing to have outsiders interfere with their content. That’s why it took a team of social scientists 5 years to get 48 U.S. news organizations to agree to run an unusual set of experiments. Instead of simply tracking what the outlets were publishing and analyzing their impact on public opinion, the researchers took an approach similar to that used in clinical trials to evaluate the effects of new drugs. They manipulated the type of news stories run, and then assigned a “treatment” week when the stories would run and a “control” week when they wouldn’t. This way they could tell whether those particular stories were having any effect on public discussion.
Most participating outlets were small, with less than an estimated 200,000 pageviews per month during this past summer, and a few were midsized, like the liberal, Wisconsin-based magazine The Progressive, which had more than 250,000 pageviews per month, according to Alexa web analytics. The nonprofit news organization Truthout, based in Chicago, Illinois, represented a large outlet, with an estimated 2 million pageviews per month.
The researchers, led by Gary King of Harvard University, asked groups of two to five of these news outlets to write stories on broad policy areas, including race, immigration, climate, and reproductive rights. For example, if the broad area was technology policy, the specific story might be what Uber drivers think about self-driving cars. The outlets could choose the policy area, the stories to cover, and the type of articles to write, such as investigative reports or opinion pieces. However, the researchers could reject a story if it was outside a specific policy area. (The outlets were free to publish whatever story they wanted outside of the experiment.)
Then, the researchers flipped a coin to decide during which of two consecutive weeks these clusters of stories, all on the same topic, would run. Finally, they measured the number of tweets about both the specific stories and the broader policy issues during the week when the stories ran compared to the week when they didn’t.
Twitter posts on these topics increased by nearly 63% over the week in which the stories were posted, the team reports today in Science. On average, Americans wrote more than 13,000 additional social media posts about a specific policy area on the day the stories ran and in the following 5 days. What’s more, the cluster of stories swayed people’s opinion by 2.3% in the ideological direction of opinion articles, suggesting that news media might in some cases change people’s beliefs.
The team repeated the experiment 35 times, and observed that stories boosted posting by men and women alike, as well as by people living in different U.S. regions, with different political orientations and influence on Twitter. Though the researchers can’t disclose the results for individual outlets for confidentiality reasons, they could show that removing larger outlets from the analysis didn’t change the effect on public conversation much, suggesting that no single large news organization was responsible for the increase.
“The results are surprising,” says economist Matthew Gentzkow of Stanford University in Palo Alto, California, who was not involved in the study. One would think that small outlets have little impact on public conversations, he says. Instead, “the effects are big.”
However, if the researchers had recruited large mainstream outlets, the spike in discussion might have been much bigger: When they looked at stories published by The New York Times on little-discussed topics, such as how fracking affects the quality of drinking water, they found that Twitter posts about the broader issue of water quality increased by 300% in just 1 day.
Though excited by the study, Gentzkow points out that only about 20% of Americans use Twitter, so the results might not be widely applicable outside social media. But to King, Twitter users are a valuable resource to assess the agenda-setting power of media because they represent those people who are willing to speak up to influence policy. And he hopes to do more experiments to find out whether collaborative projects, such as the Pulitzer Prize–winning “Panama Papers” investigation about thousands of wealthy individuals and public officials investing in offshore businesses for illegal purposes, have bigger effects on public discussion.
But whether this discussion can change people’s minds and eventually translate into political action remains an open question, says James Hamilton, an economist at Stanford who was not involved in the study. The researchers, he adds, have just started to solve a “puzzle that starts with media and ends in the voting booth.”