OpenSearch: Cluster Blocks Read-Only

[post-views]
December 12, 2024 · 2 min read
OpenSearch: Cluster Blocks Read-Only

OpenSearch can enforce read-only states on clusters or indices to protect against issues like low disk space or cluster instability. Understanding and resolving these blocks is crucial for maintaining a healthy and operational cluster. Below is a guide to address common scenarios.

How to Resolve cluster.blocks.read_only

The cluster.blocks.read_only setting typically occurs when OpenSearch detects a critical issue, such as running out of disk space. The cluster automatically sets itself to read-only to prevent further damage.
Steps to resolve:
  1. Free up disk space:
    • Delete unnecessary indices.
    • Archive old data.
    • Increase disk capacity.
  3. Clear the read-only block: Run the following command to remove the block after resolving the disk space issue:
PUT _cluster/settings
{
  "persistent": {
    "cluster.blocks.read_only": false
  }
}

Index-Level Read-Only Block

At the index level, OpenSearch can set specific indices to read-only, preventing writes. This is often triggered by disk watermarks or manual configuration.
Steps to resolve:
  1. Identify the affected indices using:
GET _cat/indices?v
  2.  Clear the block for an individual index:
PUT /<index-name>/_settings
{
  "index.blocks.read_only": false
}
  3.  If multiple indices are affected, use a wildcard:
PUT /*/_settings
{
  "index.blocks.read_only": false
}

How to Resolve cluster.blocks.read_only_allow_delete

This block allows only delete operations on indices, typically to help free up space when disk watermarks are exceeded.
Steps to resolve:
  1. Free up disk space as described earlier.
  2. Clear the read_only_allow_delete block with:
PUT _cluster/settings
{
  "persistent": {
    "cluster.blocks.read_only_allow_delete": false
  }
}

Index-Level Read-Only Delete Block

When indices are individually set to read_only_allow_delete, write operations are blocked except for deletions.
Steps to resolve:
  1. Check index settings:
GET /<index-name>/_settings
  2.  Remove the block for a specific index:
PUT /<index-name>/_settings
{
  "index.blocks.read_only_allow_delete": false
}
  3.  To remove the block from all indices:
PUT _all/_settings
{
  "index.blocks.read_only_allow_delete": false
}

Conclusion

Table of Contents

Was this article helpful?

Like and share it with your peers.
Join SOC Prime's Detection as Code platform to improve visibility into threats most relevant to your business. To help you get started and drive immediate value, book a meeting now with SOC Prime experts.
Join for Free Book a Meeting

Related Posts

OpenSearch Flush, Translog, and Refresh
Blog, Knowledge Bits — 2 min read
OpenSearch Flush, Translog, and Refresh
Oleksii K.
OpenSearch Split Index API
Blog, Knowledge Bits — 3 min read
OpenSearch Split Index API
Oleksii K.
Reindexing in Elasticsearch: A Guide for Administrators
Blog, Knowledge Bits — 2 min read
Reindexing in Elasticsearch: A Guide for Administrators
Oleksandr L