Overcoming Data Schema Complexity for Your SIEM & XDR with the SOC Prime’s Continuous Content Management Module

Security monitoring teams can bypass significant efforts tied to event data normalization by directly deploying schema-aware detection rules with the SOC Prime’s Continuous Content Management module.

Today’s cybersecurity landscape is overwhelmed with SIEM systems, EDR, NTDR & SOAR tools, next-gen XDR solutions, and innovative approaches navigating businesses through technology bottlenecks. Organizations tend to keep up with the dynamic pace of technological evolution by building a highly scalable CI/CD workflow. Data is constantly evolving as well, posing new challenges to its arrangement and management. SIEM Detection Engineers, SOC Analysts, or any security specialists leveraging SIEM or XDR have to respond to these changes by adopting new data schemas in the company’s infrastructure. 

Any industry-leading SIEM & XDR vendor is looking for ways to evolve the existing data schema and come up with a normalized solution capable of handling multiple types of data. For instance, Microsoft has adopted the Azure Sentinel Information Model (ASIM), Chronicle Security introduced the Unified Data Model (UDM) for the same purposes, while Elastic has been elaborating their common data format with each new version of Elastic Common Schema (ECS).

I have seen organizations spend 100’s of thousands of dollars on onboarding security data. Even though some vendors do not require normalization in order to collect the data, building content on top of data not adhering to a standard can produce an incredible drain on security operations teams.

Anton Goncharov

Chief Product Officer at SOC Prime

While developers are improving the SIEM and XDR data schemas, enterprises continue to collect the cybersecurity data and have to rapidly adjust to these changes. As soon as a new threat emerges, organizations should be ready to boost their detection scalability while collecting data in multiple formats. When the vendor-driven data schema fields undergo any changes, new logs are collected and parsed in the modified format. This becomes a tangible hurdle to data management turning the process of log search or threat detection with SIEM or XDR into a nightmare.

The key question concerning any SOC team in the world is how to manage data schema modifications without downtime in threat detection operations. One of the possible solutions to overcome this data schema complexity is to renormalize all the old data by adjusting it along with all implemented threat detection algorithms to the new data schema standard. However, this way of data unification still comes with downtime being both time-consuming and rather costly.

Before joining SOC Prime as Chief Revenue Officer, I was key to building a 24×7 Central-Europe Advanced Cyber Defense Center with SIEM & SOAR technology at its core protecting the biggest organizations in Europe. We implemented the latest and most advanced technologies but always had to invest an incredible number of resources in maintaining and adjusting our data schemas and detection algorithms. With the SOC Prime’s Continuous Content Management module, every SOC team can finally stop wasting days and weeks on maintenance of data schemas and focus on their main task — finding evil and protecting the organization’s core assets.

Andreas Süß 

Chief Revenue Officer at SOC Prime

The SOC Prime’s industry-first platform for collaborative cyber defense, threat hunting and threat discovery supports on-the-fly cross-tool translations for Sigma behavior-based detections using the data schema format aligned with vendor standards. When converting Sigma to one of the available 20+ SIEM & XDR formats, fields are mapped according to the vendor-driven data schema, which is native to the selected security solution. Thus, SOC content for Azure Sentinel will be automatically adjusted to the ASIM data schema by default, while the Elastic Stack translations will be mapped to the ECS format accordingly. 

What’s more, the SOC Prime’s Detection as Code platform offers non-standard data schema solutions that go beyond vendor specifications. Since SIEM & XDR vendors are constantly evolving their normalization data schemas, it’s hard to keep pace with all custom use cases and cover all possible log sources. SIEM Detection Engineers and SOC Analysts often have to process data without having the corresponding log sources covered by the vendor documentation. 

The Continuous Content Management (CCM) module available as part of the SOC Prime’s Detection as Code platform enables security practitioners to apply custom data schemas and stick to them regardless of the format changes on the SIEM or XDR vendor side. The CCM module enables pre-configuring the relevant Custom Field Mapping version before content deployment and then streaming rules and queries into the SOC environment without parsing issues. 

For instance, when collecting Windows logs via Winlogbeat, the tool automatically makes them fitting to the ECS data schema and stores them in Elasticsearch without parsing issues. Still, log collection via some non-standard tool and involving custom data that cannot be automatically mapped to the default data schema requires additional fine-tuning and involves sufficient SOC team hours. Here’s an example of the global Custom Field Mapping profile created by the SOC Prime Team that uses non-standard fields instead of the default ECS fields for Windows log collection.

With this flexible approach, one data schema can be applied to a single set of rules, and another one for a different stack of content items before deploying them via CCM jobs. In the end, you can have your own version of rules automatically customized to your environment needs. 

The CCM normalization capabilities powered by a custom data schema primarily resonate with SOC Analysts and Detection Engineers saving their precious time and a great deal of manual effort. From a business perspective, CISOs and SOC Managers can maximize their team productivity while saving sufficient costs, focusing on the most crucial SOC tasks, and reducing risks of data schema migrations.

The Continuous Content Management module and its API is now open to the entire cybersecurity community. The newly released SOC Prime’s Detection as Code platform unlocks free access to the CCM functionality, which means any platform user can take advantage of its capabilities. The only limitations are of a threshold-based nature, such as the amount of content available for download, number of content lists, filters, or presets that can be configured per user, etc. The full-scale availability to the CCM module can be reached with a paid subscription tailored to the organization’s business needs.

The CCM module currently supports the world’s leading SIEM & XDR cloud solutions, including Microsoft Azure Sentinel, Google Chronicle Security, Humio, Sumo Logic, and Elastic Cloud, enabling continuous streaming and management of Sigma detections converted to the corresponding cloud-native format and scalable to the chosen data schema. In addition to streaming detection content to the cloud, the CCM module also provides on-premises support for Splunk and Elastic Stack. Security engineers can automatically deploy new detections and update the existing content stack for their on-premises Splunk instance using the SOC Prime CCM App for Splunk. 

SOC Prime has come up with an important update by categorizing Sigma behavior-based detections that are automatically deployed to the environment into Alerts and Queries. SOC teams can now choose the detection category better fitting for their purposes — alerts that are fully tested and are less likely to generate false positives or queries of a more experimental nature to investigate or hunt for threats. This removes the ambiguity around the intended purpose of content, reduces SOC team workload, and helps streamline SOC operations.

To gain technical insights into the fully automated system of content management, read the dedicated blog post and explore the CCM module capabilities at full scale. SOC teams are welcome to see the CCM functionality in action by joining the SOC Prime’s Detection as Code platform now available for a wider cybersecurity audience. Break the limits of data complexity with minimum dependency on vendor-specific data schemas to enable hassle-free and cost-efficient content deployment and management.