Pseudonymizing Log Data in Cribl Detect with the LogTotal Sanitizer Pack

Summary

Cribl Detect, released on September 29, 2026, is a SIEM that runs on Cribl’s data platform. The data it stores and searches is ingested through Cribl Stream Routes and Pipelines, so a sanitization step in those Pipelines determines what analysts, AI-assisted triage, alert notifications and retained datasets can access.

The Cribl LogTotal Sanitizer is an open-source Cribl Pack (this project is not affiliated with Cribl or SOC Prime) built on the sanitization engine of SOC Prime’s LogTotal. It pseudonymizes free-text log data in-stream. It detects eleven categories of sensitive values and replaces each one with a keyed HMAC token that carries a type label, for example <USER:…> or <IP:…>. Tokens are deterministic, so events that reference the same user, host or address can still be correlated after the original value has been removed.

• Credentials, payment data, health data and personal identifiers are removed before data is stored. This reduces the impact of a breach and the compliance scope of the SIEM.
• Correlation-based detections continue to work on tokenized values.
• The Pack is MIT-licensed and runs on existing Worker capacity, with no per-GB licence fee.

Contact Sales

Where sanitization fits in the Detect data path

Detect uses the standard Cribl model of Sources, Routes, Pipelines and Destinations. In-stream detections, federated search, AI-assisted investigation and alert routing operate on data that Stream writes to Cribl Lake or Cribl Search datasets (SOC Prime overview). Data that Detect queries in place through federated search, such as an existing S3 bucket, does not pass through a Pipeline and has to be sanitized when it is written.

A SIEM exposes event content to more consumers than a typical log pipeline: tier-1 analysts, MSSP partners, AI agents, Slack and PagerDuty notifications, and long-term retention. Removing sensitive values once, upstream of all of them, is simpler to operate and audit than enforcing access controls on each consumer.

Origin: SOC Prime’s LogTotal

The sanitization engine in the Pack was developed by SOC Prime. SOC Prime released LogTotal as a free public preview on August 26, 2026 (announcement). LogTotal sanitizes log files locally in the browser before anything is uploaded. The sanitized events are then correlated against SOC Prime’s detection content: about one million detection rules, a dataset of 13,000 labels, Higher Order Sigma rules and agentic AI correlation. LogTotal does not retain uploaded logs.

SOC Prime published the sanitization component separately as the open-source library @socprime/logtotal-sanitizer under Apache-2.0. The Cribl Pack is a community project by M3NIX that wraps this library.

The design decisions described below follow SOC Prime’s rationale for LogTotal, which identifies three common failure modes of log redaction:

• Static masking replaces every IP address or username with the same placeholder. Ten failed logons then look identical, and a single compromised account can no longer be distinguished from a password-spray attack against ten accounts.
• Plain find-and-replace misses values in nested JSON or unusual encodings, and over-redacts values that only look sensitive, such as version numbers shaped like IP addresses or UUIDs used as message IDs.
• Unkeyed hashes can be reversed by dictionary attacks, and identical unsalted hashes from different organizations can falsely link unrelated incidents.

Keyed, typed tokens address all three.

How the Pack works

The Pack (cc-stream-logtotal-sanitizer) implements a Cribl custom Function. By default the Function applies all built-in detectors to _raw, replaces each match with a token and sets __logtotal_sanitized: true on events it modifies. _time and all other fields are left unchanged.

Detectors are evaluated in this priority order:

1. Secrets: bearer tokens, JWTs, API keys, PEM blocks and cloud provider tokens
2. Session cookies
3. Payment data, validated with Luhn and mod-97 checksums
4. Government identifiers
5. Health identifiers and ICD-like codes
6. Phone numbers
7. IPv4, IPv6 and MAC addresses
8. Hostnames and FQDNs
9. Usernames and email addresses
10. Geolocation
11. Home-directory paths

For JSON input, values under known sensitive key names are replaced based on the key name. The rest of the event is still processed by the regex detectors.

A token is the HMAC-SHA-256 of the rule ID and the original value, truncated to 16 hexadecimal characters. The same key, rule and value always produce the same token. In pseudo mode the token includes a type label. In mask mode, which is used for secrets and payment data, the token has the neutral form <R:…>. Example, with token values shortened:

before: user alice@corp.example failed login from 10.20.1.7 to db-prod-01.corp.example

after:  user <USER:3f9a…> failed login from <IP:b81c…> to <HOST:0d4e…>

Custom rules are defined as a JSON array in the Function configuration. Each rule specifies an ID, a regular expression, a mode and a token prefix. Internal ticket or employee numbers, for example, can be mapped to <TICKET:…> tokens.

Deployment

The Pack is installed with Cribl’s standard Pack workflow in the Worker Group that sends data to Detect. Release files, requirements and installation instructions are in the project’s GitHub repository. In outline, the Pack is imported with custom functions enabled, a random HMAC key is stored as a Worker Group Secret, and the output is checked against the bundled preview sample. The Pack is then set as the Pipeline of a Route that delivers to Detect’s datasets, with a filter that selects the sources containing sensitive data.

Three implementation details need attention:

• The Pack rewrites a single top-level string field, _raw by default. Fields extracted earlier in the Pipeline keep their original values, so the Pack has to run before parsing, or the fields have to be re-extracted from the sanitized _raw.
• All Workers that need to produce matching tokens must use the same key, Pack version and rule configuration. Rotating the key changes every token, so rotation should be scheduled with dataset retention periods in mind.
• Detect is available only on Cribl.Cloud. Running the Pack on a customer-managed (hybrid) Worker Group pseudonymizes data before it leaves the customer network. Cribl.Cloud has historically scoped custom functions and scripting to hybrid Workers (Cribl blog), so support should be confirmed before relying on Cribl-managed Workers.

In this layout, unmodified values exist only inside the customer network. Detections and enrichment that need the original values run before the Pack, and everything written to the destination contains tokens.

Distinguishing features

The main technical difference from Cribl’s native options is keyed pseudonymization. Original values are removed, but references to the same entity stay linkable across events.

Keyed tokens are not vulnerable to the dictionary attacks that work against plain hashes. An unkeyed SHA-256 of an IPv4 address or a username can be reversed by enumerating the small input space. An HMAC token cannot be computed without the secret key. The type label in each token (<HOST:…>, <USER:…>) still tells analysts and LLM-based triage tools what kind of entity an event refers to, so timelines remain readable.

Eleven detector families work without additional configuration. Checksum validation reduces false positives, such as random digit strings being matched as card numbers.

Re-identification does not require bulk decryption rights. Anyone holding the key can compute the token for a known indicator, such as a suspect account name, and search for it.

Because the engine is SOC Prime’s LogTotal library, it is also available as the LogTotal web application, a CLI and a Node.js package, including for air-gapped environments. With the same key and rule configuration, a log extract prepared for a vendor ticket or an incident-response retainer can be processed consistently with SIEM data. The code is open source, there is no per-GB fee, and the Pack bundles its dependencies.

Limitations and operational considerations

The largest operational impact is on detection content that depends on original values.

• Tokenized IP addresses, domains and usernames do not match IOC feeds, GeoIP databases, CIDR-based rules or asset lookups. Mitigations: run in-stream detections and enrichment before the Pack, disable the ips and hosts rules on the affected Route, or send a full-fidelity copy to a restricted store.
• The underlying library supports a neverRedact allowlist, but the Pack’s documented settings do not expose it.
• Each event is evaluated against a large set of regular expressions. Aggressive mode increases both CPU cost and false positives, and Worker sizing should account for this.
• Tokens cannot be decrypted. Re-identification is only possible by recomputing the token for a known value.

Conclusion

For deployments that send security data to Cribl Detect, the LogTotal Sanitizer Pack provides in-pipeline pseudonymization at low cost. Sensitive values are removed before they reach analysts, AI agents and long-term storage, while the entity relationships used by correlation rules are preserved. A practical rollout starts with a single high-risk source. Validate the output with the preview sample, confirm that the relevant detections still fire, and then extend the Pack to further Routes.