Elasticsearch Shards Management

readme.md

Elasticsearch Shards Management

Elasticsearch 分片管理

Before you start, curl with credentials 如果需要认证，就使用--user参数就好了

curl --user username:password -XPUT 'localhost:9200/myindex'

Explain API 解释分片

curl -XGET "http://localhost:9200/_cluster/allocation/explain?pretty

curl -XGET "http://localhost:9200/_cluster/allocation/explain?pretty"
{
    "index": "myindex",
    "shard": 0,
    "primary": true
}

Shards stats 查看分片状态

curl -XGET "http://localhost:9200/_cat/shards?v"

Move shards mannually 手动移动分片

curl -XPOST 'localhost:9200/_cluster/reroute' -d '{
    "commands" : [ {
        "move" :
            {
              "index" : "myindex", "shard" : 0,
              "from_node" : "node1", "to_node" : "node2"
            }
        }
    ]
}'

Assign stale shard as primary 指定state状态分片为主分片

Do this only if the index is RED. 只有当所有数据均损坏且无法恢复的时候，可以选其中一个作为主分片

curl -XPOST "http://localhost:9200/_cluster/reroute" -d '{
    "commands":[{
        "allocate_stale_primary":{
            "index":"myindex",
            "shard":"2",
            "node":"node_id",
            "accept_data_loss":true
        }
    }]
}'

Use Elasticsearch wrapped commandline tool to remove corrupted data and recover

$ bin/elasticsearch-shard remove-corrupted-data --index myindex --shard-id 0


    WARNING: Elasticsearch MUST be stopped before running this tool.

  Please make a complete backup of your index before using this tool.


Opening Lucene index at /var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/index/

 >> Lucene index is corrupted at /var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/index/

Opening translog at /var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/


 >> Translog is clean at /var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/


  Corrupted Lucene index segments found - 32 documents will be lost.

            WARNING:              YOU WILL LOSE DATA.

Continue and remove docs from the index ? Y

WARNING: 1 broken segments (containing 32 documents) detected
Took 0.056 sec total.
Writing...
OK
Wrote new segments file "segments_c"
Marking index with the new history uuid : 0pIBd9VTSOeMfzYT6p0AsA
Changing allocation id V8QXk-QXSZinZMT-NvEq4w to tjm9Ve6uTBewVFAlfUMWjA

You should run the following command to allocate this shard:

POST /_cluster/reroute
{
  "commands" : [
    {
      "allocate_stale_primary" : {
        "index" : "myindex",
        "shard" : 0,
        "node" : "II47uXW2QvqzHBnMcl2o_Q",
        "accept_data_loss" : false
      }
    }
  ]
}

You must accept the possibility of data loss by changing the `accept_data_loss` parameter to `true`.

Deleted corrupt marker corrupted_FzTSBSuxT7i3Tls_TgwEag from /var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/index/

Assign an empty shard as primry 分配一个空的主分片，一般不会用

curl -XPOST "http://localhost:9200/_cluster/reroute" -d '{
    "commands":[{
        "allocate_empty_primary":{
            "index":"myindex",
            "shard":"2",
            "node":"node_id",
            "accept_data_loss":true
        }
    }]
}'

There are tons of tips for shards management and deployment recommendations, please check the official documents for more details.

如何合理管理ES的分片和科学部署，还请参照对应版本的官方文档，里面有很多非常有用的信息。