What hinders successful threat hunting?

As more organizations implement successful threat hunting operations, a SANS Institute survey finds that they are facing common challenges with employing skilled staff and collecting quality threat intelligence.

successful threat hunting

“Without a sufficient number of skilled staff, high-quality intelligence, and the right tools to get visibility into the infrastructure, success with threat hunting will remain limited,” says survey author Mathias Fuchs.

“A world where we’ll see a unified, widely accepted golden standard of threat hunting remains in the future, but we are headed in the right direction.”

Key challenges in threat hunting

The survey highlights key challenges, limitations, and successes that organizations self-identify about their approach to threat hunting. Results indicate that threat hunting has arrived in the majority of organizations:

  • 65% of respondent organizations report they are already performing some form of threat hunting
  • Another 29% are planning to implement threat hunting within the next 12 months

With the concept of threat hunting being relatively new for many organizations, however, only 29% of respondents consider themselves mature or very mature in their threat hunting, with nearly 68% self-identifying their threat hunting as immature or still maturing.

Struggling to attract qualified threat hunters

Many organizations indicate that one of their top challenges is finding and employing the right experts to enable them to maintain an advanced threat hunting operation. A second main challenge respondents face is the quality of threat intelligence upon which their threat hunting is based.

Even though many organizations struggle to attract qualified threat hunters, only 21% of respondents currently outsource their threat hunting activities to external parties. Despite that, the majority of respondents rely on externally produced threat intelligence, yet only one-third of respondents claim they are highly satisfied with their sources. This presents an opportunity for organizations to improve, as well-curated threat intelligence can be leveraged to augment inexperienced threat hunters.

Measuring the benefit of threat hunting

The survey data also showed that organizations are beginning to have methodologies in place that enable them to measure the benefit of threat hunting, which bodes well for broader industry.

“Measuring the benefits of threat hunting is important,” Fuchs says. “Good threat hunting means that you probably never hear from these teams. The only indication for upper management that threat hunting even exists is that they have to foot the bill. That might be a tough sell, so if we have more ways to express the benefit of threat hunting, funding might get better, which ultimately might advance the general maturity level of threat hunting in the industry.”

Know your enemy: Mapping adversary infrastructure quickly and accurately

Group-IB is a known quantity in the information security arena: in the sixteen years since its inception, the company – now headquartered in Singapore – has detected and detailed many high-profile threats, performed over a thousand successful investigations across the globe and gained widespread recognition for helping private and public entities and law enforcement worldwide track down and prosecute cybercriminals.

To be able to do that, it has been steadily building an international infrastructure for threat detection, hunting and investigating cybercrime around the world. This infrastructure includes, among other things:

  • The largest computer forensics laboratory in Eastern Europe
  • An early warning system for proactive cyber defense based on their own threat intelligence, attribution and incident response practices
  • A certified emergency response service (CERT-GIB), which is member of the Forum of Incident Response and Security Teams (FIRST) and Trusted Introducer
  • Databases containing extensive threat and threat actor information

The company was, at the beginning, mostly a provider of digital forensics and cyber investigation services. In time, though, they realized that the solutions available to organizations were not keeping pace with the ever-morphing threat landscape, so they decided to work on and offer their own.

It all started with the creation of Group-IB Threat Intelligence (TI), an attack attribution and prediction system and service that’s based on data collected from a wide variety of sources (investigations, network sensors, honeypots, OSINT, card shops, and much more), automated information extraction and correlation technologies, and is supported by expert analysts, incident responders and investigators around the world.

It was followed by:

  • Group-IB Threat Detection System (TDS) – A threat-actor-centric (instead of malware-centric) detection and proactive threat hunting solution
  • Secure Bank – A fraud and attack prevention solution for the financial services industry, which detects threats like account takeovers, credit fraud, malicious web injections, banking trojans, remote access software, social engineering, etc. (keeps more than 100 million banking customers secure by monitoring 16 million online banking sessions every day)
  • Secure Portal – A fraud and attack prevention solution for ecommerce websites and online services (prevents account takeovers, identifies fake accounts and blocks bots, fraudulent activities, fraudulent ticket sales, and so on)
  • Brand Protection – A service designed to detect and eliminate threats to one’s brand on the Internet (brand abuse, Internet fraud, copyright infringement, counterfeiting)
  • Anti-Piracy – intelligence-driven protection of content online

Most of these solutions are powered by Group-IB TI. More recently, though, they gained another thing in common: an integrated Graph Network Analysis system for cybercrime investigations, threat attribution, and detection of phishing and fraud.

Graph Network Analysis

Many threat intelligence solutions have graph-making capabilities and the company has considered a number of graph network analysis providers before finally deciding to develop their own tool for mapping adversary infrastructure, Group-IB CTO and Head of Threat Intelligence Dmitry Volkov told Help Net Security.

None of the considered solutions gathered and used the wide variety of data and historic data Group-IB experts deem crucial for creating a complete picture for better visibility. None of them had the automated graph creation option and were able to reliably identify and exclude irrelevant results. Finally, none allowed operators to specify the ownership timeframe of the entered suspicious domain, IP address, email or SSL certificate fingerprint.

“Domain name and IP addresses change ownership – today they are used by a threat actor, tomorrow by a legitimate company or a random individual, so the timeframe within which the threat actor owned the suspicious domain name or IP address is very important information for the creation of a relevant and accurate graph,” Volkov explained.

mapping adversary infrastructure

mapping adversary infrastructure

The interface of the graph network analysis tool

The user decides how wide they want to cast the net by specifying the number of steps the tool should take when identifying direct links between elements, but the tool’s automated mode builds the graph of the links to the searched element. And, if they switch on the “refine” option, it will automatically remove from the resulting graph all the elements it deems irrelevant.

mapping adversary infrastructure

The graph network analysis tool attributing the search element to a specific threat actor

Analysts and investigators who don’t trust the tool to create a graph that contains all the crucial elements can always turn “refine” off and specify one step to build the graph themselves and then remove irrelevant elements from it.

Though, Volkov pointed out, after performing numerous manual checks and consistently seeing that the tool did a great job when allowed to do it automatically, their own experts have come to trust and prefer that option.

Improving graph accuracy

“The initial goal was just to create a useful tool for our internal analysts, and we didn’t plan to incorporate it in our products. But some of our clients saw how we were using it to do our research in-house and wanted to be able to do the same, so we decided to share it,” Volkov shared.

The company’s developers and experts have been working on the Graph Network Analysis tool for the past few years. The first version was good, but very slow. In time, they managed to improve both the speed and the effectiveness by experimenting with different types of data and different approaches to data enrichment, processing and correlation.

There are still two versions of the tool: a standalone one that’s used by Group-IB’s experts and one that’s incorporated in the company’s products. New features are first added and tested on the former, then incorporated in the latter if they prove useful.

Group-IB is constantly working on enriching the tool with data and designing new algorithms using machine learning to improve the graph’s accuracy.

“All of Group-IB’s products are being constantly fine-tuned thanks to the permanent monitoring of the cyberspace for new threats and our incident response operations and cyber investigations,” Volkov pointed out. “And we’re always analyzing existing solutions on the market, pinpointing their weak spots and shortcomings, thinking of ways to eliminate them and striving to provide the best technologies to our customers.”

The tool’s capabilities

Mapping adversary infrastructure and (hopefully) identifying the threat actor has many advantages for the targeted organization and its customers, but also for other organizations, their customers and, in general, the wider populace.

“The main goal of network graph analysis is to track down projects that cybercriminals carried out in the past — legal and illegal projects that bear similarities, links in their infrastructure, and connections to the infrastructure involved in the incident being investigated,” Volkov explained.

If the users are very lucky and a cybercriminal’s legal project is detected, discovering their real identity becomes simple. If only illegal projects are detected, that goal becomes more difficult to achieve.

But even if the identity of the attacker remains elusive, discovering details about their previous attacks can help pinpoint their preferred tactics, techniques, procedures, tools and malware, and that information can be handy for disrupting ongoing attacks or even preventing those that are yet to be launched (e.g., by identifying attacker infrastructure at the preparation stage).

The tool can be leveraged by SOC/CERT analysts, threat hunters, threat intelligence analysts and digital forensic specialists, and it’s great for improving the speed of incident response, fast cybercrime investigations, proactive phishing and global threat hunting, and pinpointing malicious servers hidden behind proxy services.

It’s also used for IoC enrichment and event correlation (i.e., discovering when certain attacks are linked and are likely different stages of a single multiphase attack).

Group-IB Graph Network Analysis was designed based on indicators of compromise discovered and collected by the company’s cybercrime investigators, incident responders and malware analysts in the last 16 years.

To this have been added or made available through data-sharing agreements and subscriptions many other data sets containing:

  • Domain registration data
  • DNS records (domain records, files, profiles, tags)
  • Service banners (domains, redirections, error codes)
  • Service fingerprints on IP addresses (which services are running and which ports are open)
  • Hidden registration data (IDs, hosting providers)
  • Historic registration data and that related to hosting transfers
  • SSL certificate registration data.

They have also made an effort to come up with new methods of extracting data that is not available using ordinary means. “We cannot reveal details for obvious reasons, but in some cases, mistakes made by hackers during domain registration or server configuration help us discover their emails, pseudonyms, or backend addresses,” Volkov said.

An advantage for all threat hunters

The tool queries both the company’s internal databases and external sources of information (e.g., WHOIS, public sandboxes, etc.) and the whole network graph creation happens in mere seconds.

And everybody wins in the scenario where the tool is used by Group-IB’s clients.

“By giving visibility to our clients, we reduce our analysts’ load and get interesting feedback from our clients. When they do the analyses themselves, they may achieve results that are more interesting and relevant to them, and when they share those results with us, we have a better understanding about the threats that target organizations in their industry, sector or geographic region,” Volkov concluded.

“This allows us to tune our research capabilities and detection engines to improve our whole ecosystem and, on a global scale, it improves our detection, prevention and hunting processes for every client.”

Cybersecurity industry predictions for 2020 and beyond

When it comes to cybersecurity industry predictions for 2020, Optiv researchers expect to see a focus on privacy, evolving threat actors, pervasive deepfake videos, and increased election interference.

cybersecurity industry predictions 2020

“As we look beyond 2019 and into 2020, we have a solid idea of what threats the industry is facing, and not just ransomware and phishing attacks, but new, hard-to-combat threats,” said Anthony Diaz, division vice president, emerging services, at Optiv.

“As is always the case, us ‘good guys’ are forced play catch up with bad actors, who constantly remain a step ahead. There is much IT and business leaders must be aware of when it comes to cybersecurity, as the pace of change is quite high.

“That is why we recommend cybersecurity programs focus on proactive risk mitigation and build out from there. This ensures your organization is actively looking for, combating, and identifying threats before they can cause damage.”

Hybrid threat actors may become more commonplace

A growing number of “hybrid threat actors” have been found. These are attackers who impersonate one type of adversary to disguise their true intentions (for example, a nation state imitating a generic hacker targeting a customer database, when its true aim is to steal intellectual property).

There could be an increase in the number of adversaries to adopt this technique and launch “imposter” attacks to obfuscate their true intentions, adding yet another layer of complexity to threat hunting and incident response.

Apple’s “privacy as a human right” campaign should cause others to follow

The world’s foremost technology organization going all-in on privacy will shift the competitive landscape. Security and privacy could become a competitive differentiator for companies that follow Apple’s lead and grab “first mover” status in their markets.

Laggards may risk meeting the unseemly fate of past organizations that failed to embrace important technology paradigms such as internet, cloud, and mobile computing.

Election misinformation campaigns could proliferate

The effectiveness of the Russian misinformation campaign of 2016 increases the possibility of increased copycat attacks for the 2020 election. These attacks could come from nation states as well as domestic groups supporting rival U.S. politicians. This activity threatens to trigger a major public/private response to the online misinformation problem.

We might see the first cases of deepfakes used to manipulate stock prices

There has been much publicity around the potential to impact elections using deepfakes (AI-doctored videos that enable individuals to make it appear people said things they never said). However, not enough attention has been paid to how cybercriminals can make money using deepfakes against businesses.

This might change in 2020, as it’s possible we will see the first deepfake attacks designed to impact stock prices, by having CEOs, financial analysts, Federal Reserve leaders or other powerful economic figures make phony statements that will cause stock market movements. Cybercriminals would use these videos to make quick fortunes in the market.

There should be widespread realignment of IT and security organizations

As boards view cybersecurity as a peer-level risk to traditional enterprise risks, such as lawsuits and product recalls, more CISOs should become peers of CIOs and other executives, rather than direct or indirect reports. This would cause a realignment of the IT and security organizations to eliminate conflicts and encourage collaboration.

The most critical of these will be the continued expansion of DevSecOps, in which security is fully integrated into the application development process; and patch management, which will move from being divided between security and IT (security finds vulnerabilities, IT patches them), to becoming a unified process with a single point of accountability.

Cybersecurity basics may continue to vex consumers and enterprise organizations

Whether insufficient passwords, lack of education and training around phishing attacks, or simple upkeep and compliance, the tiny details of cybersecurity will continue to be the cause of a vast portion of compromises if left unaccounted for.

Simple passwords (those without special characters or are extremely obvious, such as “password123”) only take minutes to crack by professional hackers and can be done inexpensively.

What is the actual role of a threat hunter?

The role and tasks of a threat hunter are confusing, according to a ThreatQuotient and SANS study based on data collected from 575 participating companies that either work with or operate their own threat hunting teams.

threat hunter role

Threat hunter role: How threat hunting teams are tasked in an environment

Unlike the Security Operations Centre (SOC) and Incident Response (IR) teams, threat hunters not only respond to network threats, they proactively search for them. This involves making hypotheses on the existence of potential threats, which are then either confirmed or disproven on the basis of collected data.

“However, the reality within corporate IT is often different,” says Markus Auer, Regional Sales Manager CE at ThreatQuotient. “In many teams, the distinction between SOC, IR and threat hunting is too blurred, and threat hunters are used for reactive processes contrary to their actual role.”

The study confirms that most threat hunters react to alerts (40%) or data such as indicators of compromise from the SIEM (57%). Only 35% of participants say that they work with hypotheses during threat hunting – a process that should be part of the arsenal of every threat hunter.

“Responding to threats is important for security, but it is not the main task of the threat hunter. They should be looking for threats that bypass defenses and never trigger an alert,” Auer emphasises.

Targeted threat discovery is important

The fact that threat hunting is still in its infancy is evident based on suboptimal prioritization of resources. “Many companies are still in the implementation phase and are more willing to spend money on tools than on qualified experts or training existing employees to be threat hunters,” says Mathias Fuchs, Certified Instructor at SANS and co-author of the study.

“When threat hunting is carried out, it is more of an ad hoc approach than a planned program with budget and resources.” In fact, 71% of participating companies consider technology to be first or second in terms of resource allocation for threat hunting. Only 47% of respondents focus on hiring new personnel and 41% on training employees.

threat hunter role

Due to the proactive nature of threat hunting, companies often find it difficult to accurately measure the economic benefits of these security measures. Ideally, the experts prevent threats from becoming a critical problem in the first place. However, 61% of respondents said their overall IT security status has improved by at least 11% due to threat hunting.

These figures show that targeted threat discovery is important and that investing in dedicated threat hunting teams delivers measurable improvement in IT security for organizations.

Cyber threats continue to evolve, but security teams remain confident

Coming off of a year of major data breaches making headline news, it’s easy to draw the conclusion that security teams are losing the cybersecurity battle, a DomainTools survey reveals. Security teams remain confident Security pros are reporting real progress being made as confidence in their programs continues to grow: Thirty percent of respondents gave their program an “A” grade this year, doubling over two years from 15 percent in 2017. Less than four percent … More

The post Cyber threats continue to evolve, but security teams remain confident appeared first on Help Net Security.