With all the recent news stories on the so-called ‘dark web’, you’d be forgiven for thinking of it as a black hole to be avoided at all costs. But what is the dark web exactly? Where did it come from and how does it actually work? And more importantly, can it ever be a force for good?
Deep web vs dark web
These terms are often used interchangeably, but they represent two very distinct concepts. The deep web is made up of content that isn’t indexed by search engines, making it inaccessible to the average user. Databases and anything hidden behind a login screen, like online banking and webmail, could be considered part of the deep web – and the vast majority of it is 100 percent legitimate and perfectly legal.
The iceberg analogy is a common way of illustrating the deep web, with easily accessible sites and social media platforms on the ‘surface web’ floating above a far larger mass of inaccessible data below the waterline. Estimates of the deep web’s size are inherently difficult, but according to some sources, as much as 96 percent of the web’s content is hidden. And unlike the real icecaps, the deep web is growing at an exponential rate.
The dark web forms a tiny part of the deep web, but while the deep web has naturally evolved into a chaotic jumble of obscure data, the dark web is enigmatic by design. Unlike the deep web in general, the dark web is based around encrypted content that can only be accessed with highly specialised software.
Shadowy origins
From the beginning, the dark web has been associated with some extremely shady, military-industrial goings-on. The concept of a ‘darknet’ was conceived in the 1970s as a way to create networks isolated from ARPANET, an early precursor of the modern internet built by the US government. Addresses on a darknet could receive data from the whole network, but did not appear in listings or answer enquiries.
The next step towards the modern dark web came with the development of ‘onion routing’ in the mid-1990s. The US Naval Research Laboratory worked on the technology alongside the Defense Advanced Research Projects Agency (DARPA), with the goal of creating a secure form of internet communication for US intelligence agencies.
Tor: a leap in the dark
Development of onion routing culminated in the Tor software (originally known as ‘The Onion Router’) in 2002. High-ups in US intelligence made a decision that at first seems totally at odds with espionage: they released the software to the public. Why? The more people using it, the harder it would be for anyone to detect US government activities amidst all the background noise.
‘The Onion Router’ refers to how Tor sends traffic through several layers of encryption to ensure maximum anonymity for the user. Requests are passed through multiple relays, with each one only able to decrypt the instructions of where to pass the request next, and nothing else. The Tor network now has more than 7,000 volunteer relays worldwide.
By routing encrypted data through a large number of servers, Tor makes it incredibly difficult to retrace the user’s steps back to their IP address, location or identity. Often used in conjunction with VPNs, Tor still can’t make the user completely anonymous, but it can drastically decrease the success rate of surveillance by third parties. As such, it’s become the tool of choice for a range of highly illegal online activities.
Criminal hideout or legitimate tool?
The most publicised uses of the dark web include the illegal trade of drugs and weapons, hosting for hacker tools and botnets, and worse. Used alongside online currencies, black markets like the now-defunct Silk Road became a haven for illicit goods, with buyers and sellers safe in the knowledge that the authorities would have a very hard time tracking them down.
While Silk Road was busted by the FBI in 2013, there are still dodgy marketplaces to be found on the dark web, highlighted by the recent takedowns of AlphaBay and Hansa. Even if it wasn’t the original intention of the Tor developers, the dark web now provides a highly effective way for criminals to hide their identities.
Still, there’s a case to be made for the legitimate use of the dark web. In 2013, Edward Snowden used Tor to leak classified details of US surveillance to The Washington Post and The Guardian. Whether you sympathise with Snowden or not, the dark web has been a useful tool for whistle-blowers, journalists, political activists and dissidents worldwide, especially as a means to circumvent repressive state censorship. And of course, even as intelligence agencies attempt to track and identify Tor users, government bodies are partially funding the software and making widespread use of it themselves.
Ultimately, the dark web is a morally grey zone with no easy answers. While it does have the potential to be used legitimately, it also provides a highly effective refuge for criminals and wrongdoing in general. Either way, the technology behind the dark web is fascinating, and a demonstration of just how far the internet can be pushed.
