How to Export Databricks Logs

As with Snowflake, “Databricks logs” can mean security data you keep in Delta tables, or Databricks’ own audit log in the system.access.audit system table. The simplest export for both is a notebook cell. Query with Spark SQL, write the result as JSON to a Unity Catalog volume, then download it with the Databricks CLI. Spark writes newline-delimited JSON, so the files are ready for upload.

Databricks is increasingly used as a security data lake, and its audit log matters in its own right: who logged in, who created tokens, who changed permissions. Prime Core has more than 24,000 unique detection rules, mapped to 676 MITRE ATT&CK techniques. Below is how to get the data out, sanitize it, and investigate it for free in LogTotal.

Get Started For Free

Before you start

Access. Reading system.access.audit needs the system schema enabled for your metastore and a SELECT grant from an account or metastore admin. For your own tables, you need the usual Unity Catalog grants.

A volume to write to. You need a Unity Catalog volume you can write to, for example /Volumes/security/exports/logtotal, and the Databricks CLI configured on your machine.

Compute. A notebook needs a cluster or serverless compute. Filter by date so you don’t scan more than you need.

Export from a notebook

(spark.sql("""
    SELECT *
    FROM system.access.audit
    WHERE event_date = '2026-10-19'
      AND action_name IN ('login', 'tokenLogin', 'generateDbToken')
""")
 .write.mode("overwrite")
 .json("/Volumes/security/exports/logtotal/audit_20261019"))

Spark writes several part-*.json files plus marker files. Download and combine them:

databricks fs cp -r dbfs:/Volumes/security/exports/logtotal/audit_20261019 ./audit_20261019
cat ./audit_20261019/part-*.json > databricks_audit_20261019.jsonl

For your own security tables, change the query: select from your table and filter on its timestamp column.

Without a notebook, the SQL Statement Execution API can run the same query on a SQL warehouse and return JSON. Rows come back as arrays, with the column names in a separate manifest, so you need an extra step to turn them into objects. For anything sizable, the notebook route is simpler.

What to look for in the audit log

Look at logins and token logins from unusual IP addresses, new personal access tokens, changes to permissions and group membership, and workspace or cluster configuration changes outside normal hours. Downloads of notebooks or results in bulk are worth a look too.

Upload, sanitize and review in LogTotal

1. Upload. Open logtotal.com and drop in the .jsonl file. 
2. Sanitize.
3. Review.

When it goes wrong