The ‘Deep Web’ is a term used to describe digital objects and data that are not accessible over the internet via search engines1)INTERPOL. The Dark Net. (2018). Available at: https://www.interpol.int/Crime- areas/Cybercrime/The-threats/The-Darknet.. It comprises protected and proprietary data (such as bank accounts, email and research data) as well as intranets. About 96% of all online data is in the Deep Web (DW), including most of the world’s Instant Messaging (IM) traffic.

A smaller fraction is known as the ‘Dark Net’, including hidden sites developed and used by people that require anonymity. The Dark Net community uses specialized platforms that obfuscate the IP address and their identity; the most popular is Tor (‘The Onion Routing’), based on software developed by the US Naval Research Laboratory in the mid-1990s2)Barlow, M. & Fell, G. Patrolling the Dark Net: What You Don’t Know Will Hurt You. (O’Reilly Media, Inc., 2016). and released in 2002. Tor is a free software that routes internet traffic through an open voluntary network of about 10,000 nodes to hidden services that use the onion address.

In 2013, an estimated 40,000 Tor sites were known3)Cole, J. Dark Web 101. (Air Force Air Command and Staff College Maxwell AFB United States, Air Force Air Command and Staff College Maxwell AFB United States, 2016).. The original designers of Tor developed it for OSINT2 (open source intelligence) collection and secure communications for Navy personnel. In 2013, the US Government invested about $1.8 million (€146 million) in Tor. Today, the Dark Net is used by criminal communities, illicit traders, journalists requiring anonymity, private citizens who want or need to circumvent censorship, private investigators and law enforcement officers. Visitors to the Dark Net typically use a VPN, Tor or similar services and one or more pseudonyms4)The Tor Project, Inc. Who uses Tor? The Tor Project (n.d.). Available at: https://www.torproject.org/about/torusers..

While the total traffic flow is unknown (users are, by definition, anonymous), estimates are indicative: The Tor project claims its traffic accounts for a very small (an estimated half a percent) of all global internet traffic5)Cole, J. Dark Web 101. (Air Force Air Command and Staff College Maxwell AFB United States, Air Force Air Command and Staff College Maxwell AFB United States, 2016).. A criminal fraction, however small, is still a dangerous fraction. About 50% of the activity in the Dark Net is considered illegal6)Cole, J. Dark Web 101. (Air Force Air Command and Staff College Maxwell AFB United States, Air Force Air Command and Staff College Maxwell AFB United States, 2016). in most jurisdictions.

“Differing national approaches to crime, combined with territorial limits on criminal jurisdiction and investigative authorities, tie legal regimes to geography in the context of an Internet and Darknet markets that are seemingly everywhere and nowhere.”—Sui et al., 20157)Sui, D., Caverlee, J. & Rudesill, D. The Deep Web and the Darknet: A look inside the internet’s massive black box. (Woodrow Wilson International Center for Scholars, 2015).

The Dark Net has become particularly known for the global illegal drugs trade (see figure 1), but it is also a significant source of leaked corporate data, access credentials and cyber weapons (see figure 2). In February 2016 an analysis of 29,532 .onion sites found that 43% were associated with leaked data, financial fraud and hacking.

Within the Deep Web, the inaccessible and anonymised Dark Net is a hinterland for cyber crime. Cyber criminals succeed because attribution is very difficult8)Rid, T. & Buchanan, B. Attributing Cyber Attacks. J. Strateg. Stud. 38, 4–37 (2015). and prosecutions too few: the timeline between crime and conviction may be many months or years. While the definition of ‘cyber crime’ depends very much on the perspective, there are crimes which cross the line: Activities that cause physical harm and losses in the real world; and at the worst, abuse of (universally accepted) human rights. Even several of the illicit Dark Net marketplaces explicitly exclude these activities9)Sui, D., Caverlee, J. & Rudesill, D. The Deep Web and the Darknet: A look inside the internet’s massive black box. (Woodrow Wilson International Center for Scholars, 2015).. The Dark Net is, either by design or accident, part of a global ecosystem that sustains crime.

Figure 1: The global digital trade in illegal drugs. Image source – The Economist10)The Economist. Shedding light on the dark web – Buying drugs online. (2016).

 

Figure 2: A February 2016 analysis by threat intelligence companies, categorising 29,532 .onion hidden services. Source: Intelliag / Darksum / Securityaffairs.co11)Paganini, P. Mapping the Dark Web searching for illegal content. Security Affairs (2016). Available at: https://securityaffairs.co/wordpress/46202/deep-web/dark-web-mapping.html..

 

As at March 2018, Darkwebnews.com lists 73 ‘markets’. The demise of the Silk Road in 2013 with the prosecution of its American creator, the takedown of Silk Road 2 shortly thereafter and the removal of two other major markets – AlphaBay and Hansa – in July 201712)Europol. Massive blow to criminal Dark Web activities after globally coordinated operation. Europol Available at: https://www.europol.europa.eu/newsroom/news/massive-blow-to-criminal-dark- web-activities-after-globally-coordinated-operation. (see figure 3, below), suggest that law enforcement agencies are increasingly successful in finding methods for combatting cyber crime – even if detections and prosecutions take months or years. Meanwhile, as an innovation marketplace for criminals, it is very hard to estimate if the Dark Net is diminishing in scale.

MarketplacesDrug ListingsTotal Listings
Silk Road 2.012,35015,925
Agora Marketplace12,05316,137
Evolution Marketplace7,23713,699
Pandora Openmarket5,2877,143
Andromeda Market (Formerly Dark Bay)2,1262,883
BlueSky Marketplace2,1262,379
Smaller MarketsDrug ListingsTotal Listings
Tor Bazaar334448
The Pirate Market333489
Black Bank Market295416
Outlaw Market (Drugs Only)123124
New Markets (Not included in our Report)Drug ListingsTotal Listings
Cloud 91,6172,284
Hydra Marketplace1,1511,510
Cannabis Road (Drugs Onlny)658658
TOM Marketplace576658
Middle Earth Marketplace243252
1776 Marketplace187209
Alpaca Maketplace138180
Dream Market3435
Total Listings46,90665,595
Table 1: Tabular display of marketplaces in the Dark Net

 

[…] criminal network structures can best be understood as complex adaptive systems […] a self-organizing network, which constantly adapts its structure and behavior according to change in the behavior of its individual components […] meaning nothing is fixed. This makes the behavior and structure […] highly unpredictable, but effective in adapting to changed environments.” —Dujin & Sloot, 201513)Duijn, P. A. C. & Sloot, P. M. A. From data to disruption. Digit. Investig. 15, 39–45 (2015).

Taken together, the Deep and Dark Net (DDW) comprise a digital world that relies on – and enables – anonymity, virtual currencies and trade in data (legal or otherwise). Like the criminal cosystems it serves14)Duijn, P. A. C. & Sloot, P. M. A. From data to disruption. Digit. Investig. 15, 39–45 (2015). , the Dark Net is continually evolving, challenging the ability of law enforcement teams to keep up15)Ellis, C. Serious and Organised Crime in the Digital Age. RUSI (2015). Available at:
https://rusi.org/commentary/serious-and-organised-crime-digital-age.
. Advanced encryption is now a mainstream technology, enabling criminal gangs to operate successful business models. Digital crime is a fixed feature of the crime landscape16)Hannigan, R. Criminal cyber gangs are disrupters in the digital economy. Financial Times (2017). . The Dark Net can also be a threat multiplier: Europol are targeting the DDW and Internet of Things (IoT) which in combination present a serious threat vector for cyber attack.

Given the apparently lawless hinterland created by the Dark Net, it is reasonable to question why it is still allowed to exist. There are three possible explanations for this:

  1. criminal networks are just too complex to track in real-time;
  2. those in favour argue that the Dark Net facilitates privacy and anonymity, particularly in places where citizens do not enjoy freedom of speech;
  3. of all the international spaces, cyberspace is probably the most difficult to police: laws that work well in one sovereign state do not easily translate to others, and – while some borders in cyberspace do exist17)Finklea, K. Dark Web. US Congressional Research Service (2015)., e.g. sites belonging to state entities, servers hosted in a single jurisdiction – cyberspace is beyond any conventional definition of human geography.

The easier target may therefore be to “focus systematically on bad actors and bad patterns”18)Sui, D., Caverlee, J. & Rudesill, D. The Deep Web and the Darknet: A look inside the internet’s massive black box. (Woodrow Wilson International Center for Scholars, 2015).. It is perhaps inevitable that technology once designed to protect is also the enabler of crimes that generate misery and human rights abuses.

The Deep Web and Dark Net: A new frontier for Competitive Intelligence?

Figure 3: Shutting down the markets. Image source: Europol19)Europol. Massive blow to criminal Dark Web activities after globally coordinated operation. Europol Available at: https://www.europol.europa.eu/newsroom/news/massive-blow-to-criminal-dark- web-activities-after-globally-coordinated-operation.

Since the time that people first sought advantage in new terrain, the tools and practice of Competitive Intelligence have informed their strategy. As a discrete business activity, Competitive Intelligence (CI) was defined in the post-war period during a time when knowledge became a valuable asset that could make and break business ventures. CI was formalised in the early 1990s20)Wright, S. Competitive intelligence, analysis and strategy: creating organisational agility. (Routledge, 2015). along with other knowledge management practices that are today experiencing their next phase of evolution in the era of advanced search algorithms and Artificial Intelligence. At core, CI is about collecting information that will guide a person or an organisation in understanding their competition and/or experimenting with a new business approach.

The advent of data sharing via the internet opened a new dimension to CI: Even before search engines indexed the global digital surface, it became possible to gain insight at-a- distance. Deep Web search solutions have been around for at least 15 years21)Bergman, M. K. White Paper: The Deep Web: Surfacing Hidden Value. J. Electron. Publ. 7, (2001).
and are well- known in the academic research community: Some of the solutions have found ready utility in the corporate sector22)Deep Web Technologies, Inc. Deep Web Tech. (2016). Available at: https://www.deepwebtech.com/.. In the past five years, companies have begun to recognise the value of the DW for CI. Germany’s Institute for Competitive Intelligence offers training in CI for the deep web23)ive Intelligence. ICI-4 Workshop: Advanced Web Intelligence (OSINT – Open Source Intelligence). Institute for Competitive Intelligence (2018). Available at: https://www.institute-for- competitive-intelligence.com/ici-workshops/ici-4-advanced-web-intelligence-osint-open-source- intelligence..

Figure 4: The Dark Net is a subset of the Deep Web: Both contain digital objects and communities that are useful to professionals in competitive intelligence and business risk management.

In the digital age of competitive intelligence, the Dark Net is also a data source – for those equipped and trained to navigate it (see figure 4). The DDW is the ‘next level’: it requires highly specialised skills, domain knowledge and ready access to legal counsel. As in cybersecurity, commercial DARKINT (dark web intelligence) is a growing and profitable field.

In March 2016, a Motherboard journalist observed that two groups of people make money from the dark web: Criminals and tracking companies who consult for law enforcement. Cyber threat intelligence companies can make about $150,000 (€122,000) per client, per year24)Cox, J. The Booming, and Opaque, Business of Dark Web Monitoring. Motherboard (2016). Available at: https://motherboard.vice.com/en_us/article/vv7b4m/the-booming-and-opaque-business- of-dark-web-monitoring. . A third group can now be added: business risk intelligence analysts.

Companies with budget and risk appetite venture into the Dark Net for:

  • Threat intelligence25)Recorded Future. Mining Threat Intelligence from the Dark Web. Recorded Future (2017). Available at: https://www.recordedfuture.com/dark-web-threat-intelligence/., analysed (with caution) as part of cybersecurity practice26)Polancich, J. The Dark Web: An Untapped Source For Threat Intelligence. Dark Reading (2015). Available at: https://www.darkreading.com/analytics/the-dark-web-an-untapped-source-for- threat-intelligence-/a/d-id/1320983. ;
  • Breach response – in cases where data has been lost and traded;
  • Counter-corporate espionage (to identify indicators of attack timelines and proprietary data searches) and business risk intelligence.
 Automated search providers, such as DarkOwl (US)27)DarkOwl. Home. DarkOwl – Darknet Big Data (2018). Available at: https://www.darkowl.com/., provide the tools and services for commercial CI in the digital age. Hedge Fund managers use the DDW to inform investment decisions28)Kinder, T. Hedge funds turn to dark web to gain an edge. (2017). Available at: https://www.fnlondon.com/articles/hedge-funds-gain-an-edge-from-the-dark-web-20170802., while specialist providers of business risk intelligence use a hybrid mix of techniques from competitive intelligence, geopolitical risk analysis, threat intelligence and DARKINT (Dark Web Intelligence) collection, see e.g. Flashpoint29)Flashpoint – BUSINESS RISK INTELLIGENCE. Flashpoint Available at: https://www.flashpoint-intel.com/. (US and global), BBCom Secure30)Business Risk Intelligence für das Deep and Dark Web. Available at: https://www.bbcomsecure.com/BRI/flashpoint-business-risk-intelligence-deep-and-dark-web.html.
(Germany and Switzerland), corma (Germany)31)Weber, J. Dark Net Investigations. corma GmbH Available at: https://corma.de/en/investigations/dark-net-investigations/.

DDW investigations and analysis is not for the uninitiated. Operations must be precise in intent and in scope to deliver business value32)Deloitte. Dark analytics. (2017).. Whether the objective is to acquire market insight, identify an insider threat or defend a companyies reputation, DDW analysis is generally not a task for the in-house information security team – however good they may be – because navigating, analysing and understanding what the DDW contains requires specialist skills that have little to do with technical expertise33)Lefkowitz, J. Why ‘DIY Deep & Dark Web Intelligence’ is a Bad Idea. Security Week (2017). Available at: https://www.securityweek.com/why-diy-deep-dark-web-intelligence-bad-idea.. Infiltrating a chat forum, for example, requires knowledge of the communication style, vernacular, online vocabulary and group dynamic; and for materials that are not in English, advanced language skills to understand not only the context and the vernacular, but also its intent. In this context, AI and complex systems analysis have an important role in the new generation of network profiling techniques34)Duijn, P. A. C. & Sloot, P. M. A. From data to disruption. Digit. Investig. 15, 39–45 (2015). .

Figure 5: Building a digital forensics team requires specialised soft skills; many of the objectives may run counter to corporate cybersecurity strategy and knowledge management practice.

Even supposing that, say, a trove of corporate data is identified for sale in the Dark Net, the seller will rarely place the entire haul in the marketplace35)Ely, A. Tracking Stolen Data on the Dark Web. IANS (n.d.). Available at: https://www.iansresearch.com/insights/reports/tracking-stolen-data-on-the-dark-web.: A subset is offered for appraisal. The intelligence analyst then has to decide if the asset is genuine and what the value to the company would be of retrieving it and/or pursuing charges against the hawker.

Getting results from research in the Dark Web is not easy and the costs can be hard to justify. It requires strict information management discipline, a working digital forensics strategy (which may not always mesh easily with the organisation’s cybersecurity strategy) and specialised skills in at least three domains: Languages and the use of vernacular in online conversations; ability to establish trust in a digital community; analytical skills to discern genuine insight from noise36)Lefkowitz, J. Why ‘DIY Deep & Dark Web Intelligence’ is a Bad Idea. Security Week (2017). Available at: https://www.securityweek.com/why-diy-deep-dark-web-intelligence-bad-idea.. The program also requires budget for on-demand legal counsel37)Ely, A. Tracking Stolen Data on the Dark Web. IANS (n.d.). Available at: https://www.iansresearch.com/insights/reports/tracking-stolen-data-on-the-dark-web. and continuous learning (see figure 5). The aim of any hunting expedition should be to produce objective evidence, that ideally will stand up to scrutiny in a court of law, while remaining accountable at every step.

This presents considerable challenges for most threat intelligence operations, because establishing ground truth of information38)Cox, J. The Booming, and Opaque, Business of Dark Web Monitoring. Motherboard (2016). Available at: https://motherboard.vice.com/en_us/article/vv7b4m/the-booming-and-opaque-business- of-dark-web-monitoring. obtained through the internet is difficult, expensive and constrained by laws in most jurisdictions.

Are you protected?

Me and my team protect you, your business and your data against cyber threats by analysing risks and developing early detection and protection concepts.

References and sources   [ + ]