What is the dark web? It sounds like a silly question in an era where the nefarious side of the internet has become part of internet folklore. Everyone thinks they have the general idea: it’s somewhere criminals hang out and sell a bewildering variety illegal goods and services away from prying eyes.

This view of the dark or ‘hidden’ web turns out to be a simplification.  The term ‘dark web’ is also a bit of a misnomer that has been talked up by journalists and security vendors who have tended to focus on negatives.

man hiding under desk istock stevecoleimages

Origins – more hidden than ‘dark’

The first question remains how the internet came to have such a space in the first place. One answer is it didn’t. Many of the things now associated with the dark web happened on public websites that few paid much attention to or weren’t indexed by search engines. That is still true today. Huge numbers of criminal sites are accessible from any web browser if you know where to look.

Nevertheless, it dawned on internet activists that even the most obscure public websites lacked credible anonymity. With state, police and commercial surveillance on the rise, it was this desire to hide users that drove the development of the hidden or ‘dark’ web, not at first the need to hide the sites themselves. The dark web, then, is primarily a place to hide internet users, keeping the list of sites they access secret. The need to hide servers emerged later.

How dark is the dark web? Accessing Tor

Building a secret internet space within the public internet was never a simple proposition and it took the development of the encrypted Tor network with US Government money in the early 2000s as a prototype privacy system to kick off what most now think of as the dark or hidden web.

Tor is basically a proxy routing system based on relays using perfect forward secrecy that make it extremely difficult to see who is looking at what from any one point in the network. To access Tor users require a Tor-enable browser or client such as Tails to access the network while servers can also be set up as ‘hidden’ using special addresses users have to know in order to visit them.  

Despite its lofty privacy ideals, Tor been accused of making bad things possible by hiding criminal activity from the police. Defenders of Tor point out that bad things have always been a part of the internet and many of its uses are perfectly legitimate in an age of massive state surveillance. As for terrorism, there is no evidence that violent extremists either use or need Tor to communicate.  

Tor is the dominant design but it has rivals such as I2P, (the Invisible internet Project), which has for a slightly different design and purpose. This has shown that it is possible to create multiple anonymous hidden webs with different characteristics.

How dark is the dark web? How big is it?

Tor and the dark or hidden web is miniscule although exactly how small depends on how you measure it. In terms of websites, it’s perhaps around 30,000 – 40,000 sites. The number of individual connections, probably a better measure, is currently estimated by the Tor project to stand at around 1.7 million per day, predominantly in the US, Russia, Germany, France and the UK, in that order.

For comparison, the public internet comprises more than a billion sites, a small but still sizeable number of which are criminal in intent. As for daily web users, this probably exceeds 2-3 billion.

The public web hosts more criminal sites than the so-called dark web.

How dark is the dark web? How do criminals use the dark web?

It’s less the amount of activity on the so-called dark web than the type of activity that attracts attention. This is undoubtedly a part of what happens there, with child porn a pioneer interest and drugs and illegal weapons sellers following close behind. Today, as far as malware criminals are concerned it would be easier to say what’s not lurking on dark web forums and marketplaces than what is from stolen data and login caches, malware-as-a-service, and compromised servers. It’s also used for professional networking among criminals.

The site that popularised the public idea of the dark web was probably Silk Road, a drugs market set up as a hidden service on Tor until it was closed by police in 2013 and its alleged founder arrested. Since then, dark markets have become so well-known many of them are advertised on the public web.

In a more recent and typical example, the xDedic market was publicised by a researcher at Kaspersky Lab. This site was used to sell around 70,000 compromised servers to professional cybercriminals.

Why is the dark web still popular?

Paradoxically, although it’s ‘dark’ to the average internet user, the dark web has never been under more scrutiny than it is today. Almost every cybersecurity firm worth its salt now devotes considerable resources to exploring and documenting what goes on there. Some specialist companies even build a lot of their business model around understanding criminal activity on the dark web, feeding that intelligence back to customers.

This is more about who uses this part of the internet than what goes on there. It is the individual criminals behind activity that intelligence firms and the police are really interested in.

Given this, it’s debatable why criminals seem wedded to a part of the web lots of experts are paid to study in great detail. The answer is probably that while the dark web is no longer as mysterious and shadowy as its nickname suggests, it does buy criminals some time. Tracking down websites hidden on it is undoubtedly slower than on the public internet. When it comes to cybercrime it is time more than clever hacking techniques that correlates to success. Today, cybercriminals use the dark web to stay slightly ahead of their pursuers even when it’s no longer always dark at all.