OpenSearch Split Index API

The Split Index API in OpenSearch is a useful feature that allows you to split an existing index into multiple smaller indices. This can be particularly valuable when you want to improve performance, scale your index, or rebalance the data without re-ingesting it.

What is the Split Index API?

• Your existing index is too large for its shard configuration.
• You need to scale out horizontally by increasing shards.
• You want to rebalance the data across more nodes for performance or capacity reasons.

Requirements for Splitting an Index

1. Read-Only State: The index must be in a read-only state.
2. Routing Allocation: You need to set the routing allocation for the split operation.
3. Multiple of Shards: The new number of shards must be a multiple of the original.

PUT /my_index/_settings
{
  "settings": {
    "index.blocks.write": true
  }
}

2. Split the Index
Now use the Split Index API to create a new index my_index_split with 4 primary shards:

POST /my_index/_split/my_index_split
{
  "settings": {
    "index.number_of_shards": 4,
    "index.routing.allocation.require._name": "node_name" 
  }
}

• my_index is the source index.
• my_index_split is the new target index.
• index.number_of_shards is set to 4 (a multiple of 2).
• index.routing.allocation.require._name ensures the split happens on a specific node (optional).

PUT /my_index/_settings
{
  "settings": {
    "index.blocks.write": false
  }
}

Validation and Confirmation

To verify the split, you can check the shard count for the new index:

GET /my_index_split/_settings

• Splitting an index is only allowed for closed indices.
• The number of shards must be a multiple of the current count.
• This is a great way to scale without reindexing or ingesting data again.

When to Use Split Index

• You initially created an index with too few shards, and now it’s underperforming.
• You want to balance your cluster’s load and improve query performance.