Last summer, while media clamored for him to comment on a scientific scandal he had helped reveal, David Broockman was keeping an explosive secret of his own. Just months earlier, he and Joshua Kalla, political scientists now at Stanford University in Palo Alto, California, and the University of California (UC), Berkeley, respectively, had revealed a study published by Science in 2014 as likely resting completely on fake data. Now, however, Broockman’s own work was confirming that the effect claimed by the fraudulent study was real after all.
The study asserted that a short interview by a gay canvasser, if done right, can powerfully reduce people’s prejudices, specifically about same-sex marriage, a “finding” that stunned social scientists. But Broockman and Kalla found discrepancies in the paper, and its lead author, political science graduate student Michael LaCour, never produced the raw data to address them. Meanwhile, the two whistleblowers had their own study underway to test the same canvassing technique with another hot-button topic: transgender people. “Journalists were still calling us about LaCour when the first wave of data were coming in from our study,” Broockman says. “It was terrifying.”
In one of the strangest twists in social science history, their study shows that the canvassing strategy really can influence biases. “The data are solid and the analysis convincing,” says Gabriel Lenz, a political scientist at UC Berkeley who was asked by the funders of the study to verify that the data were truly collected. The effect is “so large and enduring,” he says, “that many researchers will be skeptical.”
The new study and the retracted one both focus on a persuasion technique pioneered by the Los Angeles Lesbian, Gay, Bisexual, and Transgender (LGBT) Center in California, whose canvassers have conducted more than 13,000 face-to-face interviews over its nearly 50-year history. “Prejudice against transgender and gender-nonconforming people is a terrible daily reality,” says the center’s director, David Fleischer. So the canvassers aim not just to survey existing prejudices or spread awareness, but to permanently change people’s minds.
They are up against decades of research that have produced little evidence that such biases can be altered, says Elizabeth Paluck, a political scientist at Princeton University. And because attitudes toward transgender people often involve deeply held beliefs and strong emotions, she says, “many scholars would have pegged transgender prejudice as more persistent than others.” That was what made the results of LaCour’s now retracted study all the more amazing.
After trying many different persuasion techniques over the years, the LGBT Center has its canvassers follow one called “analogic perspective taking.” By inviting someone to discuss an experience in which that person was perceived as different and treated unfairly, a canvasser tries to generate sympathy for the suffering of another group—such as gay or transgender people. “We knew from our own periodic attempts at self-measurement that we appeared to be achieving strong, lasting results,” Fleischer says, but the group wanted more proof.
So the LGBT Center reached out to academic researchers to rigorously test the technique. Unfortunately for them, the first one they worked with was dishonest. The scandal was “like a big punch to our collective gut,” Fleischer says. But he knew that Broockman and Kalla were evaluating the same canvassing technique in Miami, Florida, with funding from the Gill Foundation, a Denver-based nonprofit that promotes equal rights for LGBT people.
That is why the pair examined LaCour’s results so carefully. The closer they looked, the more the study just didn’t make sense. the people who took part were interviewed at work, “but not even that many people have jobs,” Broockman says. Also, the response rate seemed unbelievably high.
With just 2 weeks to go before they launched their own study, Broockman recalls, “we were in a panic.” So they reached out to LaCour’s co-author, Donald Green, a political scientist at Columbia University who was a mentor to both Broockman and Kalla. Soon after he learned of their concerns, Green started the process that led to his voluntary retraction of the paper. (LaCour disputes that the data were fraudulent and did not consent to the retraction.)
For their version of the study, Broockman and Kalla sent 56 canvassers—some transgender, others not—to knock on the doors of 501 people living in Miami. As a control, some of the interviews focused not on transgender discrimination, but on recycling. In all cases, the 10-minute interview included a survey before and after to measure people’s attitudes regarding transgender people, as well as follow-ups ranging up to 3 months later.
The effect was as powerful as LaCour’s supposed results: The canvassing technique virtually erased the transgender prejudices of about one in 10 people, and the change lasted at least 3 months. However, Broockman and Kalla found that the interviews reduced prejudice regardless of the gender status of the canvasser, in contrast to the retracted study, which suggested that the interviewer had to be a representative of the victimized population for the change to stick.
“The findings are compelling and it will be important to see how generalizable they are in future studies,” says political psychologist Diana Mutz of the University of Pennsylvania. Opinions on the relatively new topic of transgender people, she notes, “may not be fully crystallized, thus potentially making them easier to persuade on this issue than other well-established controversies such as gay marriage.”
Green says he is pleased that the LBGT Center’s approach has been vindicated. The center “suffered a terrible blow when LaCour’s surveys turned out to be phony, as the center’s outreach efforts were written off by many as naïve,” Green says. “Now, the center has a proper scholarly evaluation of its innovative and important work.”