CODEWITHSUNDEEP
elasticsearchelasticsearch-5
kriket
kriket

Reputation: 135

Elasticsearch cluster health intermittently flaps between 'GREEN' and 'YELLOW'

We are running a 7 node cluster with "ZERO" replicas, like this:

{
"cluster_name": "my_cluster",
"status": "green",
"timed_out": false,
"number_of_nodes": 7,
"number_of_data_nodes": 7,
"active_primary_shards": 3325,
"active_shards": 3325,
"relocating_shards": 0,
"initializing_shards": 0,
"unassigned_shards": 0,
"delayed_unassigned_shards": 0,
"number_of_pending_tasks": 0,
"number_of_in_flight_fetch": 0,
"task_max_waiting_in_queue_millis": 0,
"active_shards_percent_as_number": 100.0
}

elasticsearch cluster state changes from "Green" to "Yellow" intermittently. The other interesting thing I noticed was during this intermittent cluster state changes, there is shard initializing taking place, which correlates with the cluster state changes. Is this due to the cluster running with "ZERO" replicas? What could cause the above behavior ?

Upvotes: 1

Views: 743

Answers (1)

hamid bayat
hamid bayat

Reputation: 2179

1.find that indices with 

http://IP_MASTER:9200/_cat/indices?v

2.find the node that has the shard of that indices is going assign and unassigned. 

http://IP_MASTER:9200/_cat/shards?v
  1. restart service elasticsearch on that node
  2. if problem exist you have two option.

A. lucene check index (just check that shard) 

java -cp lucene-core*.jar -ea:org.apache.lucene… org.apache.lucene.index.CheckIndex /mnt/nas/elasticsearch/graylog-production/nodes/0/indices/graylog_92/0/index/ -verbose -exorcise

if it say doesn't find the segment, try to find and cd on that path and run the command.

B. elasticsearch fix index (it check all index and is very slow)

index.shard.check_on_startup: fix

you should set this config on elasticsearch.yml of that node.

Upvotes: 1

Related Questions