CODEWITHSUNDEEP
elasticsearch
Viktor
Viktor

Reputation: 1338

Finding out on which data path shard is located in Elasticsearch

I have multiple path.datas configured for my Elasticsearch cluster. The official documentation states that only a single path is used for a single shard, so it's never splitted across multiple paths. I'd like to find a way to finding out which path on which node is used for some specific shard (primary or replica), like index my-index primary shard 0 → node RQzJvAgLTDOnEnmIjYU9FA path /mnt/data1. Tried /_nodes, /_stats, /_segments, /_shard_stores, but there are no any references to paths.

Upvotes: 1

Views: 1148

Answers (2)

glmrenard
glmrenard

Reputation: 705

Not easily but but by doing a small python script I've the info I want, here the script

import json

with open('shard.json') as json_file:
    data = json.load(json_file)

print(data.keys())
data=data['indices']

for indice in data:
  #print(indice)
  d1=data[indice]
  shards=d1['shards']
  #print(shards,type(shards),shards.keys())
  for nshard in shards.keys():
    shard=shards[nshard]
    #print(shard,type(shard))
    for elt in shard:
      path=elt['shard_path']['data_path']
      node=elt['routing']['node']
      #print(repr(elt['shard_path']['data_path']))
      #print("=========================")
      print(indice,'\t',nshard,'\t',node,'\t',path)

They you obtain stuff like

log-2020.11.06 1 oxx /datassd/elasticsearch/nodes/0

log-2020.11.06 0 oxx /datassd/elasticsearch/nodes/0

log-2020.11.05 1 oxx /datassd/elasticsearch/nodes/0

Upvotes: 0

Val
Val

Reputation: 217254

You can find that info using the indices stats API by specifying the level=shards parameter

GET index/_stats?level=shards

will return a structure like this

  "indices": {
    "listings-master": {
      "primaries": {
        ...
      },
      "total": {
        ...
      },
      "shards": {
        "0": [
          {
            "shard_path": {
              "state_path": "/app/data/nodes/0",
              "data_path": "/app/data/nodes/0",
              "is_custom_data_path": false
            },
            ...
          }
       ...

Upvotes: 6

Related Questions