How to Export Logs from Elasticsearch and Kibana

Use a short Python script with the official client’s helpers.scan, or the _search API with a point in time (PIT) and search_after. For quick pulls of up to 10,000 rows, ES|QL with format=json works straight from Kibana’s Dev Tools. Kibana Discover can generate a report too, but not in a format LogTotal reads today. If your integration preserved it, event.original holds the raw log line.

Elastic is one of the most common foundations for in-house security analytics, from Elastic Security to self-built stacks. Prime Core has 23,722 detection rules for Elastic, mapped to 671 MITRE ATT&CK techniques. Below is how to get data out, sanitize it, and investigate it for free in LogTotal.

Get Started For Free

Before you start

Authentication. Create an API key in Stack Management → API keys with read and view_index_metadata on the indices you need. Send it as Authorization: ApiKey <key>. On Elastic Cloud, the Elasticsearch endpoint is on your deployment’s overview page.

Know where the data lives. Elastic Agent integrations write to data streams named logs-<dataset>-<namespace>. Linux authentication logs from the System integration, for example, land in logs-system.auth-default.

Check for event.original. Many integrations don’t keep the raw line by default; it’s a “preserve original event” option. If it’s missing, the parsed fields are what you have, and that’s fine for a JSON upload.

Data tiers. Searches against cold or frozen tiers are slower. Keep windows tight when the data is old.

The examples follow the current Elastic documentation, checked October 2026.

Option 1: Python with helpers.scan (recommended)

import json, os
from elasticsearch import Elasticsearch, helpers

es = Elasticsearch("https://my-deployment.es.example.com:443", api_key=os.environ["ES_API_KEY"])
query = {"query": {"range": {"@timestamp": {
    "gte": "2026-10-19T00:00:00Z", "lt": "2026-10-20T00:00:00Z"}}}}

with open("elastic_auth_1019.jsonl", "w") as out:
    for hit in helpers.scan(es, index="logs-system.auth-*", query=query, size=1000):
        out.write(json.dumps(hit["_source"]) + "\n")

For raw lines, write hit[“_source”].get(“event”, {}).get(“original”, “”) instead of the whole document.

Option 2: ES|QL for quick pulls

In Kibana Dev Tools:

POST /_query?format=json
{
  "query": """
    FROM logs-system.auth-*
    | WHERE @timestamp >= "2026-10-19T00:00:00Z" AND @timestamp < "2026-10-20T00:00:00Z"
    | KEEP @timestamp, host.name, user.name, source.ip, message
    | LIMIT 10000
  """
}

Two catches. ES|QL returns 1,000 rows unless you set LIMIT, and it tops out at 10,000. The JSON response also comes as columns plus values, not as records. To turn it into one object per line, save the query body as query.json and run it through jq:

curl -sS -H "Authorization: ApiKey $ES_API_KEY" -H 'Content-Type: application/json' \
  "https://my-deployment.es.example.com/_query?format=json" -d @query.json \
  | jq -c '.columns as $c | .values[] | [$c, .] | transpose | map({(.[0].name): .[1]}) | add' > esql_1019.jsonl

Option 3: Export logs from Kibana UI (Discover)

For smaller investigations, Kibana Discover lets you find, filter, and export logs directly from the browser without writing scripts.

1. Open Discover. Sign in to Kibana and navigate to Analytics → Discover, or select Discover from the main navigation.
2. Select your data. Choose the data view containing the logs you need, such as logs-system.auth-*. Make sure it includes the relevant Elasticsearch indices or data streams.
3. Set the time range. Use the time picker in the upper-right corner to select the investigation period. Narrow the window to avoid exporting unnecessary events.
4. Filter the logs. Use the search bar and available filters to isolate relevant events. For example, enter event.category: authentication to focus on authentication activity.
5. Select the fields. Add useful columns such as @timestamp, host.name, user.name, source.ip, and message. If available, include event.original to preserve the raw log content.
6. Export the results. Open More → Export → Tab results as CSV (in some versions, Share → CSV Reports). Generate the report and download the CSV file. If the export is not downloaded immediately, find it under Stack Management → Reporting.

Alternative: Copy logs as JSON. For a small selection of events, select the relevant rows in Discover, open the Selected menu, and choose Copy documents as JSON. Paste the copied content into a text editor and save it as a .json file. Only the fields included in the current table selection may be copied.

Limitations. CSV reports are intended for moderate-sized exports and have row-count and file-size limits that depend on the Kibana version and configuration. Exporting a Discover session definition as JSON does not export the log events themselves.

For LogTotal, use a supported log format. When JSONL is required, export the records using the Python or Elasticsearch API methods described below, or convert the exported data into one JSON object per line before uploading.

Step 3: Upload, sanitize and review in LogTotal

1. Upload. Open logtotal.com and drop in the .jsonl file.
2. Sanitize. Look it over before you continue.
3. Review. 

When it goes wrong