SIEM Impact, Pain, Actionability, and Severity
Table of contents:
Purpose
The purpose of this blog post is to introduce the metrics (Pain, Actionability, SIEM Impact, and Severity) that have been introduced to SOC Prime’s Threat Detection Marketplace.
Introduction
SOC Prime’s Threat Detection Marketplace enhances your security operations with quality detection content.
As with all defensive technology deploying all possible content “out of the box” is not recommended, and choosing what content provides the most value can sometimes be difficult. With the new metrics SIEM Impact, Pain, Actionability, and Severity, SOC Prime hopes that deciding which rules are a fit for you will become more straightforward.
However, these metrics do not directly reflect the quality of a content item. A rule can potentially have a low score in most of the metrics but still be a quality detection.
- SIEM Impact: Is the anticipated performance impact on a SIEM
- Pain: The score reflects where the rule falls on the pyramid of pain. Higher pain rules are ideal for threat hunters and L3 operators.
- Actionability: How obvious is triage. A high actionability score reflects that a L1 analyst will be able to quickly understand the next steps. A lower actionability rule may take advanced steps such as memory forensics, reaching out to the account owner, or performing many additional searches.
- Severity: How critical is reviewing any alerts caused by this logic. This is derived from the SIGMA level field.
Pain, Actionability, and Severity are provided by the analyst who created the rule in a good faith effort to reflect on the qualities of the rule. SOC Prime reviews all content and can adjust values based on our expert knowledge and customer feedback. SIEM impact is currently assigned via semi-automated means, and this will be fully implemented in the future.
In the following sections I will describe how these items are scored and treated within the Threat Detection Marketplace.
Obtaining the Metrics
When you select a rule in the search panel, metrics about the rule will be populated on the right side of the screen. These metrics will be available to community and exclusive content.
SIEM Impact (1 – 3, higher is less impact)
SIEM impact is the anticipated impact on the system you will run the rule against.
SIEMs are analytics systems requiring massive amounts of computational power to achieve a positive outcome. This makes a task of tuning every bit and byte of the SIEM configuration a daily routine. People who have spent years in SIEM and detection engineering or are familiar with regex best practices know that a rule with an event=/.*threat.*/ statement is a ‘hungry’ rule. Add one of them into a big production installation and you may experience a slower search. Add a hundred, and even a distributed system will start losing performance. We have observed various SIEM vendors providing hungry statements on training as a best practice, to put emphasis on how easy it is to find threats. In production it is important however to make cost efficient rules and this is exactly why we’ve introduced this metric.
As the complexity of the rule and the amount of noise in the data source(s) increase the lower the SIEM impact score will be.
| Score | Logic | Reasoning |
| – | message contains “a” OR “b” OR “c” OR “d” OR “e” OR “f” OR “g” OR “h” OR “i” OR “j” OR “k” OR “l” OR “m” OR “n” OR OR “o” OR “p” OR “q” OR “r” OR “s” OR “t” OR “u” OR “v” OR “w” OR “x” OR “y” OR “z” OR “1” OR “2” OR “3” | This rule would not be accepted as it is too impactful. |
| 1 | user = “*$” AND (commandline contains “TVqAA”) (commandline contains “http://” OR commandline contains “https://” OR commandline contains “ftp” OR commandline contains “file:\\\\”) | Lots of wildcard/contains usage |
| 2 | eventid=4688 AND commandline contains “scrobj.dl” | Use of contains increases load on SIEM. |
| 3 | eventid:5140 AND sharename:Admin$ | Exact match on fields is optimal from a performance perspective |
_
Pain (1 – 3, higher is more behavioral/less easy to bypass)
The Pain score is an abstraction of where the rule fits on David Bianco’s Pyramid of Pain and how easily bypassed the logic is. The more behavioral and less likely the rule is to be bypassed the higher the pain score will be. Behavioral rules are generally better for threat hunting and often may be noisier or require things like stack counting to be effective. IOC based rules should cause less noise but may be very rare to trigger or only useful for historical analysis. The logging mechanism being disabled or subverted entirely is not taken into account with this score (i.e. tools like ghost in the logs)
| Score | Logic | Reasoning |
| 1 | destination.ip=”1.1.1.1″ AND destination.port=53 | This is an IP based detection, IPs are easily changed. |
| 2 | eventid=4688 AND commandline contains “scrobj.dl” | This is a behavior detection but scrobj.dll can be renamed to avoid this detection. |
| 3 | eventid:5140 AND sharename:Admin$ | If someone connects to an admin share on Microsoft this log is not easily bypassed |
_
Actionability (1 – 3, higher is more obvious)
The Actionability score will reflect how “digestible” the rule is. Often, this is a direct reflection on the log source itself. The more context available in the original log source, the higher the actionability score is likely to be. Again, this is based on the alert data itself. Actionability of all rules in your environment may increase greatly via automation and orchestration. This is not taken into account.
| Score | Logic | Reasoning |
| 1 | destination.ip=”1.1.1.1” AND destination.port=53 | Depending on the data source this alert could be very difficult to triage. Often rules like this trigger on hosts that are not managed (guest network, etc). Additionally DHCP can introduce complexities in identifying the offending host. |
| 2 | eventid:5140 AND sharename:Admin$ | The windows event id 5140 will provide the username and hostname/ip that accessed a share. However, additional searching will likely be required to identify if this is the result of a compromise. |
| 3 | eventid=4688 AND commandline contains ”scrobj.dl” | The windows event id 4688 provides a lot of immediately useful context to an analyst. It is likely that identifying a probable compromise is possible by only viewing this alert. |
Severity (1 – 3, higher is a more critical alert)
Severity is an abstraction of the SIGMA levels (low, medium, high, and critical).
| Score | SIGMA Level | Reasoning |
| 1 | Low | Please see: https://github.com/Neo23x0/sigma/wiki/Rule-Creation-Guide |
| 2 | Medium | |
| 3 | High & Critical |
Conclusion
Finding a match for content appropriate to your needs should be as simple as possible. By introducing the SIEM Impact, Pain, Actionability, and Severity metrics, SOC Prime hopes to help match content by our Threat Bounty Program developers with the requirements of your defensive operation.
If you have an internal detection engineering team we recommend that they implement these concepts and capture them as a metric during defensive content creation.
Meta
Published – April 2020
Last Updated – 15 April 2020
Authors – Adam Swan (@acalarch) | Andrii Bezverkhyi (@andriinb)
