Making Use of Fillnull and Values() to Increase Rule Resiliency in Splunk

Within splunk we use “stats” and “tstats” a bunch as threat hunters. However, these useful operations can cause interesting events to be dropped unexpectedly.

For instance:

index=windows
sourcetype=*winevent* AND
EventCode=4688 AND
NewProcessName=*Evil.exe
| stats count by ComputerName, ParentProcessName, NewProcessName, CommandLine

CommandLine is a field in 4688 events that needs to be enabled via “group policy”. So, Will cause all events from hosts without process auditing enabled to be dropped from the output as “stats” will drop any event with an empty field on the right hand side of the “by”.

To fix this problem, we can throw a fillnull and not miss events.

index=windows
sourcetype=*winevent* AND
EventCode=4688 AND
NewProcessName=*Evil.exe
| fillnull ComputerName, ParentProcessName, NewProcessName, CommandLine
| stats count by ComputerName, ParentProcessName, NewProcessName, CommandLine

Additionally, with datamodels and tstats this issue remains. However, we can’t do a fillnull on a datamodel. So, it is imperative that everything on the right side of a tstats OR is guaranteed to be there (e.g. _time).

Generally, it is most resilient to pair _time with another very common field that provides a unique event such Alerts.id in the Alerts datamodel. You can always also use the fields in your WHERE clause if you use it on the right side, as that field is generally required for the intention of the search to work at all (you won’t miss anything).

All of the other fields you may need for contextual purposes can be left on the left side of the tstats OR and renamed as below.

| tstats values(Alerts.signature) as signature values(Alerts.severity) as severity values(Alerts.host) as host count from datamodel=Alerts BY _time, Alerts.id

Make this part of your template!