What is Cyber Threat Hunting? The Ultimate Guide
Table of contents:
Cyber Threat Hunting is a novel approach to Threat Detection which is aimed at finding cyber threats within an enterprise’s network before they do any harm. This includes deliberately looking for weak spots as well as any signs of ongoing attacks within a digital infrastructure. Threat Hunting is more complex than passive Threat Detection and requires specific processes, solutions, and expertise.
Identifying sophisticated cyber-attacks is not easy so let’s dig deeper and try to understand what is Cyber Threat Hunting in a real SOC environment and how it works. Beyond Threat Hunting definition, we’ll talk about a common routine that every Threat Hunter exercises on a daily basis.
Before we dive into learning, make sure to check out the Threat Intelligence timeline for the threats of your interest in our Cyber Threats Search Engine. Once you have all the necessary info at hand, you can access Threat Hunting queries at our Detection as Code platform, modify them in a browser, and enhance your hunting experience by leveraging seamless integration with numerous security environments that you might be using.
Detect & Hunt Explore Threat Context
How to Start Threat Hunting?
The first steps in proactive Threat Hunting often define the overall success. Put simply, to find a threat, you need to have an idea of what to look for. Imagine an average organization that generates roughly 20,000 events per second. That’s 1,728,000,000 events per day. Then, there are over 450,000 malware samples registered daily. Automated algorithms are not an ideal solution when it comes to finding thoroughly hidden threats and new strains of malware. So, how many cybersecurity people do you need to cope with such big data? The truth is, there are never enough security specialists unless they narrow down their threat research to what really matters. So let’s see how to identify the main direction of Threat Hunting.
Situational Awareness
At this point, Threat Hunters need to be aware of system architecture, network infrastructure, and asset configurations. A fine-tuned visibility into the organization’s digital ecosystem allows it to tactically and strategically move on to further steps.
Situational awareness means that Threat Hunters know potential targets of attackers, as well as the current level of protection. When it comes to the elements of the environment, hunters also view those “within a volume of time and space, the comprehension of their meaning, and the projection of their status into the near future,” as defined by the OODA loop concept by Col. John Boyd. To have that kind of visibility, a correct configuration of security tools and solutions is extremely important. For example, without logging command lines and PowerShell scripts, it’s impossible to identify many kinds of severe attacks.
Threat Landscape & Attack Surface Awareness
Cyber Threat landscape is a big picture of all the cyber threats that currently exist and might be dangerous. Many of them are assumed by Threat Hunters (we’ll talk about this in the next section) because there is not enough information about how they operate, what the goals are, etc. Yet, they might be invisible to the researcher’s eye. In some sources, you’ll find that the threat landscape is a list of all the known threats, but this view is limited. It’s better to acknowledge that some part of a threat landscape is still in the shadows, yet nevertheless, it exists. And Threat Hunters usually work with that shadow part. Another distinctive feature of a threat landscape is that it’s dynamic. It mutates and develops due to numerous circumstances; that’s why it’s important to always keep track of the news, intelligence, and latest research.
Attack surface is the overall number of vulnerabilities (both known and zero-day), potential misconfigurations, and anomalies in the organization’s digital infrastructure. Since today many software applications have numerous dependencies and are often deployed on cloud servers, it’s barely possible to define a network perimeter as such. Hence, the attack surface increases. At the same time, if you take a printer manufactured 20 years ago and connected to one personal computer with a perfectly patched Windows and without the Internet connection, it has a smaller attack surface. Of course, it’s understandable that the attack surface increases exponentially with the complexity of the network. And let’s not forget that threats exist even when there are no vulnerabilities. That’s why NIST recommends building digital infrastructures that are secure by design.
Risk Prioritization
There are many approaches to creating and maintaining cybersecurity risk management. Organizations employ frameworks from NIST, ISO 270001, DoD, and more. All in all, it’s about defining the risks that are applicable to a particular business context, prioritizing them, defining a risk appetite, documenting mitigation playbooks, and regularly reviewing their efficiency.
What does risk management have to do with Threat Hunting? It’s where everything starts. For example, out of 100% risky patterns, 50% are patched, 30% are not applicable due to network configuration, 10% are handled by automated security solutions, and 5% are remediated manually. So what’s left is 5% of the unknown risks. That’s where Threat Hunting begins.
What are Threat Hunting Steps?
Threat Hunters are the ones who face unknown threats, so once they have something suspicious to work with, they begin the Threat Hunting steps. Identifying the suspiciousness itself also takes a lot of domain knowledge and experience. For example, something that looks like a DDoS attack might be just multiple computers on the network booting at the same time. So, to avoid going in the wrong direction, every Threat Hunting step should be well thought-out. In general, we can outline three steps.
Generate a Threat Hunting Hypothesis
The Threat Hunting hypothesis comes first, just like in any other type of research work. Dive into our guide on Cyber Threat Hunting hypothesis examples if you want to know more. Remember, even if your hypothesis proves wrong, you still get a valuable piece of information for your further research. For example, you assumed that some of the rare HTTP user agents were malicious, but then you found out that they weren’t. It’s fine, now you have a better situational awareness of what’s happening within the company. By the way, challenging your hypotheses and comparing them with alternative ones can be more useful than a “satisficing” approach when you only want to prove yourself right and don’t recognize the missing parts.
Perform Tests & Analysis
Now that the hypothesis is made, it’s time to test it. Threat Hunters might use different Threat Hunting tools as there are multiple ways of testing their hypotheses. They can search for specific behavior in system logs, test malware samples in an emulated environment, look at network data flow, and more. The hardest part is to find a way of detecting what you’re looking for. Let’s say if you want to perform a beaconing detection, but your network utilizes DNS over HTTPS encryption, there are some tricks you should know. Finding malicious signals, in this case, is possible, but some system configurations might prevent you from a successful search, resulting in false negatives.
The analysis part is the most interesting as there are multiple ways of working with data. Sometimes it’s better to go for mathematical algorithms like factor analysis, but sometimes it’s better to visualize the threats. There are lots of threat modeling tools. For instance, TypeDB is used by Threat Hunters quite often. Also, you can go for graphs, charts, and diagrams which are especially fun if you want to find things like outliers. If you can program distribution graphs, standard deviation, box plots, scatter plots, isolation forests and find patterns, this type of analysis could reap some great results.
However, everything automated rarely works well with non-numeric data. Machine learning is good for identifying differences without diving into context (maybe except for Naïve Bayes, which is good for textual content).
At some point, it’s necessary to add some human element to Threat Hunting. When this time comes, try the Diamond Model of Intrusion Analysis and TTP-based hunting. If manual review would take ages because of a large pool of data, tools like Clearcut might be helpful. Then, know your users’ habits and add a dash of inspiration. Learning subjects like statistics, data science, or even cognitive psychology will enrich your hunting experience.
Remediate
No matter how good it feels just to be a free hunter and a creative researcher, at the end of the Threat Hunting pipeline, some important responsibilities come into play. When you’ve got the analysis results, it’s time to document and act on them to prevent the threat you found from doing any harm. In large organizations, this thing called Incident Response gets passed on to SOC members who do this on a regular basis while Threat Hunters go back to hypotheses and analysis.
Known vulnerabilities can be patched. This part is perhaps an easier one. However, it’s challenging to watch after all kinds of patches on all kinds of assets. Some of the latter don’t like patching at all, like heavy-loaded servers which have to work 24/7, and any maintenance gives fluctuations in business operations. Such patches should be properly planned. Otherwise, SOC teams can explore zero downtime patching (ZDP) options.
Tough cases need a unique approach, so they are usually handled manually. Overall, the incident response may include lots of different actions. Threat Hunting certification training usually includes theory and practice on those. Beyond effective remediation, it’s sometimes necessary to perform data recovery. As part of proactive threat defense, SOC members are enhancing the protection of the systems based on Threat Hunters’ predictions. If a threat is benign, they might as well do nothing about it.
Threat Hunting Service
Outsourcing services is quite common in the IT field, so why not outsource Threat Hunting services? If hiring security specialists from outside is beneficial for a long-term strategy, helps to take some excessive workload off the internal team, and delivers much value for the negotiated price, then it’s wise to go for it.
A lot of vendors offer Threat Hunting services on top of their software, like IBM, CrowdStrike, Verizon, ESET, and Palo Alto Networks. SOC Prime Security Engineers provide SIEM audit with MITRE ATT&CK® coverage. They can help you identify configuration issues, errors, and limiting factors across various security products that you might be using. After the initial audit, they can perform managed security services, such as:
- SIEM cleanup
- Storage cost reduction
- Content tuning
- Log source filtering & roadmap
- Performance optimization
- MITRE ATT&CK® alignment
- Architecture sizing and holistic cost optimization
- Hard skills training plan
- Technology augmentation
- Process transformation
- Budget and stakeholder alignment
- Evolution roadmap
All these enhancements act on different levels, from operational to strategic, helping Threat Hunters to detect hidden threats.
Types of Threat Hunting
Threat Hunters recognize three different types of Cybersecurity Hunting to share responsibilities and accomplish their goals more efficiently. Threat Hunting techniques might be the same across various types of Threat Hunting or differ depending on the complexity of the security CI/CD pipeline.
Structured
Structured Threat Hunting is based on the Indicators of Attack (IoA), which are in turn based on MITRE ATT&CK® Adversary Tactics, Techniques, and Procedures (TTP). This is the top of David Bianco’s pyramid of pain. The point of such a granular view on kill chains is to hunt down attackers before they know it (i.e., cause them a lot of pain).
TTPs are fun because one technique inevitably causes the other. It’s like if you found Discovery, look for Execution. If you found Execution, look for Persistence, and so on. If a certain technique has been leveraged, you certainly know what data sources you need and in which places you’ll go inside these sources. In general, structured Threat Hunting is good for detecting zero-day exploits when there are no sure-shot indicators.
Unstructured
A suspicious event or series of events might act as triggers for launching unstructured hunting. To gain more context, a Cyber Threat Hunter wants to gather all sorts of information. What happened before and after the trigger? What else happened? Do those events correlate, and how? They want to answer all these questions.
Mind that indicators of compromise (IoC) can stretch much further than URLs, domain names, and IP addresses. In fact, many types of anomalous events are quite tangible. You can get their identifiers via threat intel and/or from the internal logs.
For the full list of what threat intel can offer, check the list of STIX cyber-observable objects. As for the inside sources, for example, Sysmon logs processes, sessions, network connections, etc. These logs include GUID (globally unique identifiers). Hence, a nicely made Threat Hunting rule can operate identifiable values that represent a compromise like:
- Anomalous database read volumes
- Suspicious inbound and outbound traffic
- Suspicious changes in system files
- And much more
As you’ll see, anything anomalous is either too little or too big. Too large command lines, too small (obfuscated) strings, too many DNS names with too many numbers and letters, and so on. Usually, security engineers automate the detection of all these by setting up baselines and thresholds in the SIEM. So, why would they bother Threat Hunters, you would ask? Because it’s surprising how easily IOCs like that get overlooked. For instance, suspicious domain names go through a proxy DNS server, and the SOC team simply doesn’t see them. That means it’s time to hunt.
For added efficiency, Threat Hunters can divide into Tier 1,2,3 just like analysts. While the first group does real-time cybersecurity Threat Hunting, the second one explores TTPs, and the third one works on advanced analysis with ML tools, math, and data science. Depending on an enterprise’s Threat Hunting Maturity Model, this defense tactic may or may not be viable.
Situational
Situational Threat Hunting can originate from two major types of sources: internal and external. Internal sources include things like regular risk assessment, Jewel in the Crown analysis, and other considerations of your unique infrastructure and traffic. External sources are Threat Intelligence, news, vulnerability feeds, and research findings.
While enterprise-level Threat Intelligence is a primary source of situational threat awareness, alternative ones are also trending. For example, Twitter is good for knowing what’s going on in the world of cyber. Check out these accounts:
Beyond Twitter, there are many other sources worth exploring. Open-source Threat Intelligence pulls tons of interesting data that’s worth a hunt. Additionally, new and hot queries are always available in a Quick Hunt module at SOC Prime Detection as Code platform. They don’t require extensive preparation and experience, which is good for junior Threat Hunters.
What’s the Difference Between Threat Hunting and Threat Intelligence?
Put simply, Threat Intelligence is useful information, and Threat Hunting is making sense of that information. Threat Hunters are proactively searching for adversaries within an organization’s network. They use Threat Intelligence feeds as an input to trigger their hunting processes.
Some Threat Hunting platforms like SOC Prime Detection as Code include Threat Intelligence context together with other types of useful functionality like the ability to discover and edit rules and then deploy them in a SIEM, EDR/XDR, or SOAR environment. It’s necessary to continuously monitor Threat Intelligence feeds because they contain the latest information about cyber threats that were discovered across different private and public networks.
Why is Threat Hunting Important?
Sophisticated threats are not that easy to detect. Whether it’s about living off the land or evading detection for months, modern malware has its ways of bypassing traditional security controls. Threat Hunting cyber security is important because it improves the visibility of advanced malware and helps to avoid the damage that it might cause. Preemptive Threat Hunting is officially suggested by CISA and the FBI.
Threat Hunters contribute to the improvement of cybersecurity defenses overall because they act as threat researchers. By applying a whole spectrum of technical and scientific knowledge, hunters decompose malware samples and stages of kill chains to understand how they work. And when such understanding is gained, it becomes possible to develop more sophisticated methods of threat detection and response.
Join SOC Prime Detection as Code platform to access thousands of Cyber Hunting queries that anticipate the newest attacks. Power up your SOC pipeline with Sigma-based rules that are translated to numerous vendor-specific formats and can instantly be deployed into a SIEM environment. And if you have your own expert knowledge, you are encouraged to share your detection items in our global crowdsourcing initiative, Threat Bounty Program, where Security Engineers and Researchers enhance collaborative defense and get repeated payouts for their effort.
