Read our COVID-19 research and news.

Millions of people have downloaded apps designed to alert them to coronavirus exposure. Their effectiveness has yet to be proved.

Minzayar Oo/Panos Pictures/Panos Pictures/Redux

COVID-19 contact tracing apps are coming to a phone near you. How will we know whether they work?

Science’s COVID-19 reporting is supported by the Pulitzer Center.

Last month, when the Australian government launched a smartphone app called COVIDSafe to find and alert the contacts of people infected with the coronavirus, Prime Minister Scott Morrison made a hard sell to the public. “The more people who download this important public health app … the sooner we can safely lift restrictions and get back to business and do the things we love,” he said in a 26 April press release. Two million people downloaded it in the first 24 hours it was available.

The idea is that such digital contact tracing will identify people potentially exposed to the coronavirus who should self-isolate—and that they’ll voluntarily do so. But so far, we only have epidemiological models to suggest such apps can help control an epidemic. Skeptics worry the apps will amount to a high-tech distraction. And even some advocates say they’re only as strong as a health system’s ability to follow up with notified users, test them, and offer support during quarantine.

“Its very appealing—that you have an app that does all this work … and does it quickly,” says Hannah Clapham, an epidemiologist at the National University of Singapore Saw Swee Hock School of Public Health. But she warns that an app can’t replace the work of human contact tracers. “I hope that it works. But I worry that we think its going to save us.”

Australia was among the first countries to launch a national contact tracing app, and many more plan to do so in the coming weeks. As health departments weigh competing app designs and prepare their pitches to privacy-conscious citizens, they’ll have to define success. It will be hard to prove an app has slowed the rate of infections and changed the course of an epidemic. But teams of epidemiologists, engineers, and behavioral scientists have many ways to put them to the test.

Emerging apps of many flavors

App-based contact tracing is appealing in part because the coronavirus’ spread is so stealthy. Infected people can transmit the virus for days before they develop symptoms, and it can take several more days for public health investigators to learn about a case and confirm it with testing. These teams then have precious little time for traditional contact tracing: interviewing the infected person, tracking down all the recent contacts they can recall, and getting those people to self-isolate before they, too, pass on the virus.

Local health departments, many of them understaffed, are straining to keep up. “By the time you get the data, you have a couple days to chase people down,” says C. Jason Wang, a health policy researcher at Stanford University who is working with health departments on their COVID-19 response. But if smartphones could detect when two users are close enough to share the virus, an app could alert one person as soon as the other gets sick—even if those people are strangers who just happened to sit in adjacent subway seats. “The technology response is absolutely necessary,” Wang says, “and it needs to be fast.”

There are different proposals (and plenty of debate) about what information an app should gather and how much it should share with health officials. The Chinese government has taken phone tracking to an extreme, monitoring citizens’ locations and purchases to gauge their risk and restrict their movement. GPS data from phones can identify potential hot spots and indicate who has been exposed. Government programs in South Korea, India, Iceland, and U.S. states including North Dakota and Utah are using phone location data to monitor COVID-19’s spread. But GPS technology isn’t precise enough to gauge short distances between two phones to determine which encounters are most risky. And widespread, automated GPS tracking raises privacy concerns that could lead to legal challenges in some countries.

Many governments are instead developing apps that identify recent contacts by the exchange of low-energy Bluetooth radio signals. Each phone generates a random numerical ID that it broadcasts to nearby phones, which record such Bluetooth “handshakes.” If a user experiences symptoms or tests positive, they can trigger notifications to phones they’ve recently been near.

Does it recognize risk?

A fundamental test is whether phone sensors can pick out the kinds of interactions likely to spread the virus. An app must estimate the distance between two people based on the strength of the Bluetooth beacon from one phone when it reaches the other. Researchers are now running experiments to gauge how walls and other obstacles attenuate the signal.

“It looks very messy,” Douglas Leith, a computer scientist at Trinity College Dublin, says of recent data he collected using a version of Singapore’s Bluetooth-based TraceTogether app. He and Trinity computer scientist Stephen Farrell found that when people sat across a table from each other, signal strength was much lower if their phones were in their pockets than if they set the phones on the table. Sometimes, the strength of the signal increased as people moved farther apart—potentially because of reflection off of metal surfaces such as supermarket shelves. (The results have been posted online but not yet peer reviewed.) Leith worries Bluetooth-based apps will fail to alert people of risky encounters and flood them with false alarms.

But imperfect Bluetooth readouts can still be useful if theyre interpreted conservatively, says Marcel Salathé, an epidemiologist at the Swiss Federal Institute of Technology Lausanne. He is advising the Swiss government on an app launching in a pilot phase this month that aims to detect when someone comes within 2 meters of an infected person for at least 15 minutes. He thinks his team can tune the system so that “if somebody gets an exposure notification, we will feel damn sure its actually been a contact.”

Does it move the masses?

An app can only catch interactions between people who have installed it, so public buy-in is key to success. Singapore, which pioneered app-based contact tracing with its launch of TraceTogether on 20 March, now reports more than 1.4 million users—roughly one-quarter of the country. But this far-from-universal uptake in a nation with strong public trust in the government makes some researchers skeptical about widespread adoption elsewhere. Uptake of an app needs to be almost improbably high to really capture all of the contacts that might be relevant,” says Allison Black, a genetic epidemiologist at the University of Washington, Seattle, and the Fred Hutchinson Cancer Research Center.

Recent modeling by infectious disease epidemiologist Christophe Fraser and colleagues at the University of Oxford predicted how the use of an app might reduce a virus’ reproduction number, a representation of how many people catch the virus from each infected person. They found that if about 56% of the population (or about 80% of all smartphone users) used an app, it alone could reduce that number from about three (roughly where it was at the start of the epidemic) to below one (the threshold for controlling the outbreak). The model assumed that people over age 70 remained in lockdown, but that no traditional contact tracing was underway, and that widespread social distancing rules weren’t in place.

Salathé says the result has led to “clearly false” suggestions in the media that an app would work only if at least 60% of the population uses it. An app can still prevent infection and save lives at much lower levels of uptake, he says: As soon as you have double digits, I think the effect is already quite substantial.”

Of course, to slow the coronavirus, people need to do more than just download the app—they need to stay home when it tells them to. Part of the job of a contact tracer is to connect a person to social supports that help them self-quarantine, for example by arranging grocery delivery or even a hotel stay, says Michael Reid, an infectious disease doctor at the University of California, San Francisco. He is working with the citys public health department to hire and train contact tracers.

Ideally, anyone instructed by an app to self-isolate could access the same supports. But because most apps keep users anonymous, health officials won’t automatically know who gets an alert or what they need, Wang says. An app can advise the contacts of an infected person to call their local health department for advice or to arrange testing. But Wang doesn’t expect every app user to dutifully check in as soon as they get an alert. “That’s too optimistic,” he says. “We tell people to stay at home; they go to the beach.”

Does it find the newly infected?

To test whether an app is catching disease transmission, health officials will also want to know what proportion of the contacts identified through the app end up getting sick—the so-called secondary attack rate. Data from traditional contact tracing in Shenzhen, China and in South Korea have estimated that about 15% of contacts within an infected person’s household get infected. That number—alongside the virus’ estimated reproduction number of two to three—suggests more than half of transmission is happening outside households, says Jessica Metcalf, a demographer at Princeton University. An app could aim to catch some of this community transmission that traditional contact tracing might miss.

But depending on app design, health officials may lack access to data on who an infected person has been close to. Some apps—including Australia’s COVIDSafe and the U.K. National Health Service’s app, launched in a pilot phase this month—have a centralized design, which means the infected person uploads both their own phone’s ID code and the phone IDs of their recent contacts to a central server. Although these IDs are anonymized, officials can see the entire network of contacts.

In Norway, which launched its centralized app in April, municipalities will compare how many contacts are identified and how fast these contacts get alerted by the app versus through traditional contact tracing, says Emily MacDonald, an epidemiologist at the Norwegian Institute of Public Health. (A drop in the country’s reported cases—to 10 to 20 per day—has made it difficult to test the app so far, she says.)

Advocates of centralized apps say the design makes it easy to check whether the right people are getting notifications. Researchers can see all the phones that got an alert and whether those users later reported symptoms or a positive test through the app. Officials can also analyze Bluetooth handshakes that didn’t lead to a notification because, for example, the contact was deemed too short, notes Fraser, who is advising the U.K. government on its app. If too many unnotified users (or not enough of the notified ones) get sick, the app needs tuning.

Other apps, such as those under development in Switzerland and Germany, will be decentralized, meaning that data about a phone’s recent interactions stay on that phone. An infected user uploads only their own anonymized ID to a central database; all phones with the app regularly load the list of infected users to check for a match with phones they’ve recently been near. Privacy advocates see big advantages to this design and argue that it doesn’t leave data about users’ social networks vulnerable to hacking or exploitation. (Google and Apple yesterday released technology that will allow a Bluetooth tracking app to run in the background of their devices—making operation smoother and sparing battery life—but only for official government health department apps that adopt a decentralized design.)

But with a decentralized app, health departments and researchers only learn about people who actually call in to report getting an alert. They can’t see how many notified people they might be missing, which could make it harder to evaluate the app’s accuracy and precision. Still, health departments can compare the attack rate for contacts they learn about through traditional interviews and through an app, says Salathé, who is part of a team developing an international protocol for managing data in decentralized apps. As a rule of thumb, if the app’s attack rate matches or exceeds that of the traditional method, “we know the app is doing a really good job.”

Does it break the curve?

Some researchers are considering how to put apps through randomized trials, to see whether they are directly responsible for bringing down infections. “I’m not saying it’s impossible,” says Johannes Abeler, a behavioral economist at Oxford, but “it would be very costly and difficult.” Because COVID-19 remains relatively rare, such studies might need tens of thousands of participants to see statistically significant differences in the number of infections between an app-using group and a control group, he says.

Researchers could also try a retrospective approach: comparing infection rates among people who have already opted for and against downloading an app. But people with a higher risk of infection might be more motivated to use an app, Abeler says, which could swamp any potential benefit it provides. To make matters more confusing, an app isn’t intended to protect its user, but that user’s contacts. So the most relevant question for an efficacy study wouldn’t be “did you download the app?” but “what proportion of your contacts downloaded the app?” That would be nearly impossible for a participant to answer.

One possibility is to compare changes in infection rates between geographic areas or demographic groups with different levels of app use, suggests Rosalind Eggo, an epidemiologist at the London School of Hygiene & Tropical Medicine. She’s not involved in building or developing apps, but she hopes to study their impact. “We have a lot of technology that can help us here,” she says. “There’s quite a lot of people saying, Oh, it wont work.’ I think we need to try.”