Events correlation plays an important role in the incident detection and allows us to focus on the events that really matter to the business services or IT/security processes.Splunk software supports a lot of ways to correlate events, such as:
⢠event correlations using time and geographic location;
⢠transactions;
⢠subsearches;
⢠field lookups;
⢠joins.
In this article, we will consider the use of event correlation based on field lookups and joins. In my opinion, this is one of the lightweight ways to correlate events unlike the use of subsearches or joins in one search query. In most cases, we need to compare one field from an event with the appropriate field from another event to search for matches. For example, we try to detect suspicious activity in our network and to learn who performs the scan for TCP 445 in our network and tries to connect to C&C servers.
Let’s start with search, which will help us to detect scan for TCP 445 in our network.
Find all events with connections to the 445 port:
Criteria for detecting scan is: one host scans 30 hosts in 1 minute, so using bucket and eventstats it is not hard to group events and find count greater than 30:
As a result, we detect that host 10.10.10.3 performed scan for 763 hosts per 1 minute in our network:
The host should be added to the list of suspicious hosts. To do this, we need to run search and put results in a lookup table:
Result:
This search automatically creates lookup suspicious_hosts.csv with fields src_ip, HostsScanned,_time.
Now we need to find out who in our network tried to connect to Ransomware C&C:
Note. IP Addresses used in the article are not necessarily Ransomware C&C at the time of writing the article.
Consolidation of results from both searches in one:
Result:
We used correlation scenario to detect an infected host in our network. For full automation, you can put these requests into Alerts. Using lookup significantly increases the efficiency because it is much easier to compare events with a static table than to make subsearch again and again.
Thank you for your attention,
Alex Verbniak
Name | Descripiton |
---|---|
PHPSESSID | Preserves user session state across page requests. Cookie generated by applications based on the PHP language. This is a general purpose identifier used to maintain user session variables. It is normally a random generated number, how it is used can be specific to the site, but a good example is maintaining a logged-in status for a user between pages. |
sp_i | Used to store information about authenticated User. |
sp_r | Used to store information about authenticated User. |
sp_a | Used to store information about authenticated User. |
Name | Descripiton |
---|---|
tuuid | Collects anonymous data related to the user's visits to the website, such as the number of visits, average time spent on the website and what pages have been loaded. |
tuuid_last_update | Collects anonymous data related to the user's visits to the website, such as the number of visits, average time spent on the website and what pages have been loaded. |
um | Collects anonymous data related to the user's visits to the website, such as the number of visits, average time spent on the website and what pages have been loaded. |
umeh | Collects anonymous data related to the user's visits to the website, such as the number of visits, average time spent on the website and what pages have been loaded. |
na_sc_x | Used by the social sharing platform AddThis to keep a record of parts of the site that has been visited in order to recommend other parts of the site. |
APID | Collects anonymous data related to the user's visits to the website. |
IDSYNC | Collects anonymous data related to the user's visits to the website. |
_cc_aud | Collects anonymous statistical data related to the user's website visits, such as the number of visits, average time spent on the website and what pages have been loaded. The purpose is to segment the website's users according to factors such as demographics and geographical location, in order to enable media and marketing agencies to structure and understand their target groups to enable customised online advertising. |
_cc_cc | Collects anonymous statistical data related to the user's website visits, such as the number of visits, average time spent on the website and what pages have been loaded. The purpose is to segment the website's users according to factors such as demographics and geographical location, in order to enable media and marketing agencies to structure and understand their target groups to enable customised online advertising. |
_cc_dc | Collects anonymous statistical data related to the user's website visits, such as the number of visits, average time spent on the website and what pages have been loaded. The purpose is to segment the website's users according to factors such as demographics and geographical location, in order to enable media and marketing agencies to structure and understand their target groups to enable customised online advertising. |
_cc_id | Collects anonymous statistical data related to the user's website visits, such as the number of visits, average time spent on the website and what pages have been loaded. The purpose is to segment the website's users according to factors such as demographics and geographical location, in order to enable media and marketing agencies to structure and understand their target groups to enable customised online advertising. |
dpm | Via a unique ID that is used for semantic content analysis, the user's navigation on the website is registered and linked to offline data from surveys and similar registrations to display targeted ads. |
acs | Collects anonymous data related to the user's visits to the website, such as the number of visits, average time spent on the website and what pages have been loaded, with the purpose of displaying targeted ads. |
clid | Collects anonymous data related to the user's visits to the website, such as the number of visits, average time spent on the website and what pages have been loaded, with the purpose of displaying targeted ads. |
KRTBCOOKIE_# | Registers a unique ID that identifies the user's device during return visits across websites that use the same ad network. The ID is used to allow targeted ads. |
PUBMDCID | Registers a unique ID that identifies the user's device during return visits across websites that use the same ad network. The ID is used to allow targeted ads. |
PugT | Registers a unique ID that identifies the user's device during return visits across websites that use the same ad network. The ID is used to allow targeted ads. |
ssi | Registers a unique ID that identifies a returning user's device. The ID is used for targeted ads. |
_tmid | Registers a unique ID that identifies the user's device upon return visits. The ID is used to target ads in video clips. |
wam-sync | Used by the advertising platform Weborama to determine the visitor's interests based on pages visits, content clicked and other actions on the website. |
wui | Used by the advertising platform Weborama to determine the visitor's interests based on pages visits, content clicked and other actions on the website. |
AFFICHE_W | Used by the advertising platform Weborama to determine the visitor's interests based on pages visits, content clicked and other actions on the website. |
B | Collects anonymous data related to the user's website visits, such as the number of visits, average time spent on the website and what pages have been loaded. The registered data is used to categorise the users' interest and demographical profiles with the purpose of customising the website content depending on the visitor. |
1P_JAR | These cookies are used to gather website statistics, and track conversion rates. |
APISID | Google set a number of cookies on any page that includes a Google reCAPTCHA. While we have no control over the cookies set by Google, they appear to include a mixture of pieces of information to measure the number and behaviour of Google reCAPTCHA users. |
HSID | Google set a number of cookies on any page that includes a Google reCAPTCHA. While we have no control over the cookies set by Google, they appear to include a mixture of pieces of information to measure the number and behaviour of Google reCAPTCHA users. |
NID | Google set a number of cookies on any page that includes a Google reCAPTCHA. While we have no control over the cookies set by Google, they appear to include a mixture of pieces of information to measure the number and behaviour of Google reCAPTCHA users. |
SAPISID | Google set a number of cookies on any page that includes a Google reCAPTCHA. While we have no control over the cookies set by Google, they appear to include a mixture of pieces of information to measure the number and behaviour of Google reCAPTCHA users. |
SID | Google set a number of cookies on any page that includes a Google reCAPTCHA. While we have no control over the cookies set by Google, they appear to include a mixture of pieces of information to measure the number and behaviour of Google reCAPTCHA users. |
SIDCC | Security cookie to protect users data from unauthorised access. |
SSID | Google set a number of cookies on any page that includes a Google reCAPTCHA. While we have no control over the cookies set by Google, they appear to include a mixture of pieces of information to measure the number and behaviour of Google reCAPTCHA users. |
__utmx | This cookie is associated with Google Website Optimizer, a tool designed to help site owners improve their wbesites. It is used to distinguish between two varaitions a webpage that might be shown to a visitor as part of an A/B split test. This helps site owners to detemine which version of a page performs better, and therefore helps to improve the website. |
__utmxx | This cookie is associated with Google Website Optimizer, a tool designed to help site owners improve their wbesites. It is used to distinguish between two varaitions a webpage that might be shown to a visitor as part of an A/B split test. This helps site owners to detemine which version of a page performs better, and therefore helps to improve the website. |
Name | Descripiton |
---|---|
_hjid | Hotjar cookie. This cookie is set when the customer first lands on a page with the Hotjar script. It is used to persist the random user ID, unique to that site on the browser. This ensures that behavior in subsequent visits to the same site will be attributed to the same user ID. |
_hjIncludedInSample | This cookie is associated with web analytics functionality and services from Hot Jar, a Malta based company. It uniquely identifies a visitor during a single browser session and indicates they are included in an audience sample. |
intercom-id-[xxx] | This cookie is used by Intercom as a session so that users can continue a chat as they move through the site. |
intercom-session-[xxx] | Used to keeping track of sessions and remember logins and conversations. |
demdex | Via a unique ID that is used for semantic content analysis, the user's navigation on the website is registered and linked to offline data from surveys and similar registrations to display targeted ads. |
CookieConsent | Stores the user's cookie consent state for the current domain. |
__cfduid | Used by the content network, Cloudflare, to identify trusted web traffic. |
ss | These cookies enable the website to provide enhanced functionality and
personalisation . They may be set by us or by third party providers whose
services we have added to our pages. These services may include the Live Chat facility, Contact Us form(s), the Product Quotation forms and submission process, and the Email Newsletter sign up functionality . |
Name | Descripiton |
---|---|
_ga | This cookie name is asssociated with Google Universal Analytics - which is a significant update to Google's more commonly used analytics service. This cookie is used to distinguish unique users by assigning a randomly generated number as a client identifier. It is included in each page. Registers a unique ID that is used to generate statistical data on how the visitor uses the website. request in a site and used to calculate visitor, session and campaign data for the sites analytics reports. By default it is set to expire after 2 years, although this is customisable by website owners. |
_gat | Used by Google Analytics to throttle request rate. This cookie name is associated with Google Universal Analytics, according to documentation it is used to throttle the request rate - limiting the collection of data on high traffic sites. It expires after 10 minutes. |
_gid | This cookie name is asssociated with Google Universal Analytics. This appears to be a new cookie and as of Spring 2017 no information is available from Google. It appears to store and update a unique value for each page visited. Registers a unique ID that is used to generate statistical data on how the visitor uses the website. |
IDE | Used by Google DoubleClick to register and report the website user's actions after viewing or clicking one of the advertiser's ads with the purpose of measuring the efficacy of an ad and to present targeted ads to the user. |
r/collect | Used by Google DoubleClick to register and report the website user's actions after viewing or clicking one of the advertiser's ads with the purpose of measuring the efficacy of an ad and to present targeted ads to the user. |
test_cookie | Used to check if the user's browser supports cookies. |
collect | Used to send data to Google Analytics about the visitor's device and behaviour. Tracks the visitor across devices and marketing channels. |
ads/user-lists/# | These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. |
c | Registers anonymised user data, such as IP address, geographical location, visited websites, and what ads the user has clicked, with the purpose of optimising ad display based on the user's movement on websites that use the same ad network. |
khaos | Registers anonymised user data, such as IP address, geographical location, visited websites, and what ads the user has clicked, with the purpose of optimising ad display based on the user's movement on websites that use the same ad network. |
put_# | Registers anonymised user data, such as IP address, geographical location, visited websites, and what ads the user has clicked, with the purpose of optimising ad display based on the user's movement on websites that use the same ad network. |
rpb | Registers anonymised user data, such as IP address, geographical location, visited websites, and what ads the user has clicked, with the purpose of optimising ad display based on the user's movement on websites that use the same ad network. |
rpx | Registers anonymised user data, such as IP address, geographical location, visited websites, and what ads the user has clicked, with the purpose of optimising ad display based on the user's movement on websites that use the same ad network. |
tap.php | Registers anonymised user data, such as IP address, geographical location, visited websites, and what ads the user has clicked, with the purpose of optimising ad display based on the user's movement on websites that use the same ad network. |