Most of the world wide web is invisible. Beyond the “surface web”—the parts accessible to search engines—there is a “deep web” containing (by one estimate) 500 times the content, secured in databases and hidden behind login screens. And within this deep web is a tiny corner known as the “dark web,” which requires special, anonymizing software such as the Tor Browser to access and contains everything from black markets selling drugs and counterfeit IDs to whistleblowing forums.
Researchers have just conducted a comprehensive mapping of the dark web and found that it’s not much of a web at all. They started with a few central hubs in the “.onion” domain (sort of like .com on the surface web) and used an algorithm to crawl along links from site to site, finding only 7178 sites, connected to each other through 25,104 links. (Sites with no inbound links couldn’t be counted.) Their key finding is that 87% of these dark web sites don’t link to any other sites. The dark web is more of a set of “dark silos,” they write in a preliminary paper posted on arXiv yesterday. Dark websites linked to surface websites and to other dark websites at the same rate, ruling out dark sites’ ephemerality as an explanation for their scant interconnections.
“I personally find this rather strange, and interpret it as, socially speaking, people who create dark web sites are just less social beings,” says Virgil Griffith, as computer scientist at the Singapore-MIT Alliance for Research and Technology and the paper’s lead author. “Or at the very least, the dark web is barely used as a social mechanism—while the world wide web most definitely is.”