What Is Detection Engineering?

Threat detection engineering (DE) is more complex than it might seem initially. It goes far beyond the detection of events or abnormal activities. The DE process includes detecting states and conditions, which is often more applicable to incident response or digital forensics. As Florian Roth mentions in his blog, the definition of detection engineering “should be considered work-in-progress.” These security operations require expertise in understanding the tactics, techniques, and procedures employed by threat actors and translating that knowledge into effective detection strategies.

Detection engineering is about developing an environment inside an organization where several teams collaborate to address risks and target potential threats better. Detection engineering involves dealing with designing, developing, testing, and maintaining threat detection logic. This threat detection logic can be a rule, a pattern, or even a textual description. The scope of DE is wide and works in multiple dimensions, from risk management to threat intelligence. 

  Source: SANS 

Although threat detection engineering is an increasingly popular term in the information security sector, it is still a relatively new concept in its developing phase. The on-hand information we have available is rather limited, and there aren’t that many specialists dedicated to this particular area of cybersecurity. Yet, numerous companies are testing out their methods to use this theory and enhance their detection and response capacities.

One of the important concepts of detection engineering is Detection-as-Code (DaC). Essentially, DaC means that detection will involve the best implementation practices of software engineering by using the modern agile CI/CD (continuous integration and continuous delivery) pipeline.

What Are the Benefits of Detection Engineering?

Security engineers utilize the DE process for creating, testing, and tuning detections to alert teams about any malicious activities and minimize false positive alerts. Some benefits of detections engineering are as follows:

Dynamic threat identification: Advanced techniques and tactics help detection engineers to identify risks at an early stage of the attack lifecycle. In addition to classic signature-based detection techniques, it also discovers dynamic and sophisticated threats in real time. This proactive approach aids in spotting possible dangers before they cause serious harm.

Improved incident response: Activities related to incident response are made easier by automation and adaptability employed in detection engineering. Quicker response times are made possible by automated systems that can quickly analyze and prioritize security events. The ability to customize response actions based on the kind and degree of detected threats allows for the optimization of incident mitigation efforts.

Reduced dwell time: The amount of time it takes to find a security breach is known as dwell time. By immediately recognizing and reacting to threats, detection engineering seeks to shorten the breach discovery period. By utilizing comprehensive techniques and routinely modifying the detection parameters, organizations can significantly reduce the amount of time that exploits go unnoticed within their networks.

Improved threat intelligence: By using detection engineering, organizations can incorporate threat intelligence feeds and data sources into their detection architecture. This relationship enhances detection capabilities by providing the most recent information on new threats, attack patterns, and indicators of compromise. The usage of threat intelligence can help security teams to stay ahead of emerging risks and make informed choices during incident response. With SOC Prime’s Uncoder AI that combines the power of collective industry expertise and augmented intelligence, teams can instantly explore relevant CTI, including false-positive, triage, and alert configuration.

Why is Detection Engineering Important?

Security analysts work closely with alerts and data provided by security tools like SIEM, EDR, and XDR,  to detect suspicious and abnormal activity. The main problems with these detection methodologies are the following:

Security teams can develop scalable writing and hardening detection methods to identify complex threats quickly by implementing DaC. Much like well-written application code, detection should also be treated as well-written code that can be tested. Security teams can provide better warnings and identify malicious behavior by checking source control and reviewing it with peers. 

Features of a Good Detection Engineering Process

The threat detection engineering process requires both good tooling and excellent detection content to work with. Yet, detection engineering isn’t only about complex queries. Even simple rules count as long as they can be saved, reused, and allow automatic alerting. A good DE process has many key features for identifying and responding to security alerts.

The whole process is broken down into a single graphical view by the SANS whitepaper. You have to build your strategy and repeat it continuously because detection engineering is a cyclic process. Here are the components essential for an effective DE process: 

Source: SANS whitepaper 

Detection engineering: DE begins with a clear picture of the data an organization generates. The next step is to assess the organization’s inventorying capabilities and data sources for event generation. Only by knowing the data sources can engineers understand the detection possibilities. The trick here is to find a balance between having broad detections that generate a large volume of false positives and having a precise rule with a high level of false negatives. So every query should be tested for alert volume and accuracy.

Detection maintenance stage: After deploying any detection rule, you have to maintain it. As any code requires updates and bug fixes, the same works for the queries. The best approach is to generate as much feedback as possible to improve the detection capabilities of a rule collectively. At SOC Prime, we trust in collective cyber defense as it benefits the whole industry. 

Threat intelligence: Threat intelligence is what feeds detection engineering because we could define DE as the process of transforming threat context into detections. Threat detection engineering can’t be fully efficient without proper data generation and collection. That’s why threat intel is integral to DE.

What Role Does an EDR/XDR Tool Play in Detection Engineering?

The EDR (endpoint detection and response) tool is designed to protect the network’s endpoints, whereas the XDR (Extended Detection and Response) tool is used to simplify network security management by enforcing security policies across the network.

Both tools, EDR and XDR, work as security analysis engines. These tools can detect threats automatically using internal and external network knowledge and threat intelligence. These detections can also be set up to automatically respond to certain events, for example, terminating a process (shutting down a machine) or sending alerts to the security team.

The threat landscape is constantly evolving. EDR/XDR platforms may contain a large number of detection rules; they are meant to provide a solid foundation for the majority of settings. No set of regulations or rules applies to everything. That’s why we can’t rely only on those rules and standards. Different networks and organizations have distinct environments, and detection must conform according to the setup and compliance for every particular case. Key points that an organization needs to include in its detection and response tool are:

The Role of Threat Intelligence in Detection Engineering 

Threat intelligence is the process of collecting data to analyze motives and identify the targets of threat actors. A security analyst with intel on internal and external data can build good detection rules. Threat intelligence platforms provide new updates and feed about developments in the cyber threat landscape. They also offer threat context to enrich and enhance the detection process.

Proper detection engineering requires contextual data based on tactics, techniques, and procedures (TTPs) described in the MITRE ATT&CK® framework or other frameworks. Context is essential because it simplifies many processes, as attackers tend to use the same kill chains. Threat actors often take the path of least resistance to achieve their objectives. That’s why analyzing different attack vectors is a good idea when it comes to threat hunting and detection.

Even though the most effective indicators for detection are TTPs (tactics, techniques, and procedures), indicators of compromise (IOCs) are also important in detection engineering. But IOCs are much less flexible than the behavioral-based detection rules. Often IP addresses, domains, and hashes won’t be reused by the threat actors. Besides, if you’re dealing with a time-sensitive exploit, indicators of compromise might not be published yet. Still, IOCs are effective for retrospective detection to discover if you were targeted in the past. 

Apart from detection engineering, organizations should also leverage threat hunting to strengthen their security posture. Without threat hunting, DE isn’t complete and might end up being less efficient. Focusing on known threats is essential, yet that alone can’t ensure a complete defense system. Of course, cybersecurity is the industry where 100% protection is a mere myth. But the more comprehensive your approach is, the less vulnerable your systems will end up being. 

By leveraging SOC Prime’s Uncoder AI, teams are equipped with the augmented intelligence framework for advanced detection engineering that fuses the power of Sigma and ATT&CK and relies on peer-driven expertise to code, validate, and share detection ideas faster and simpler than ever before.