Understanding index.mapping.total_fields.limit in OpenSearch/ElasticSearch

[post-views]
December 10, 2024 · 2 min read
Understanding index.mapping.total_fields.limit in OpenSearch/ElasticSearch

Sometimes, you can get the associated error Limit of total fields [1000] has been exceeded

I will explain what it is and how to fix it.
You can find that error in OpenSaerch/ElasticSearch logs /var/log/opensearch or /var/log/elasticsearch
For example, in the screenshot, you can see that error:

In OpenSearch and Elasticsearch, the number of fields in an index is governed by the index.mapping.total_fields.limit setting. This parameter sets the maximum number of fields allowed in an index mapping, and exceeding this limit will cause indexing operations to fail.
Default Limit:
The default value for index.mapping.total_fields.limit is 1,000 fields per index. This includes:
  • Explicitly defined fields in the mappings.
  • Dynamic fields that are created during document indexing.
Adjusting the Limit:
If you need more fields, you can increase the limit by updating the index settings. However, be cautious, as having too many fields can impact cluster performance (e.g., higher memory usage and slower queries).
Update the limit in Dev-Tools:
  1. For an Existing Index:
PUT /your-index-name/_settings
{
  "index.mapping.total_fields.limit": 2000
}
2. If you want to update the limit automatically when creating an index, use this index template for your index:
PUT _template/your-template-name
{
  "index_patterns": ["*"],
  "settings": {
    "index.mapping.total_fields.limit": 2000
  }
}

Was this article helpful?

Like and share it with your peers.
Join SOC Prime's Detection as Code platform to improve visibility into threats most relevant to your business. To help you get started and drive immediate value, book a meeting now with SOC Prime experts.
Join for Free Book a Meeting

Related Posts

Standard Logstash Template for Event Processing (Gold Template)
Blog, Knowledge Bits — 3 min read
Standard Logstash Template for Event Processing (Gold Template)
Oleksandr L
Configuring Disk Allocation Thresholds in Elasticsearch and OpenSearch
Blog, Knowledge Bits — 3 min read
Configuring Disk Allocation Thresholds in Elasticsearch and OpenSearch
Oleksii K.
Monitoring Elasticsearch Cluster With Metricbeat
Blog, Knowledge Bits — 2 min read
Monitoring Elasticsearch Cluster With Metricbeat
Comrade H.