CODEWITHSUNDEEP
javaelasticsearch
Stephane Karagulmez
Stephane Karagulmez

Reputation: 177

Data volume on Elasticsearch cluster

I have a simple elasticsearch cluster and I've seen into the documentation that the master node has to access to the elastic data volume.

But the fact is that if two nodes use the same data volume this error occured : Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/elasticsearch/data/escluster]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?

I've tried many configuration but can't find how to share the volume between my different nodes.

Docker-compose 

version: '2'

services:

  esmaster:
    build: elasticsearch/
    volumes:
      - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    volumes_from:
      - esdata
    environment:
      cluster.name: "escluster"
      node.data: "false"
      http.enabled: "false"
      node.master: "true"
      ES_JAVA_OPTS: "-Xmx2g -Xms2g"
      discovery.zen.ping.unicast.hosts: esmaster
    cap_add:
      - IPC_LOCK
    networks:
      - elk

  esclient:
    build: elasticsearch/
    volumes:
      - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    ports:
      - "9200"
      - "9300"
    environment:
      cluster.name: "escluster"
      node.data: "false"
      http.enabled: "true"
      node.master: "false"
      ES_JAVA_OPTS: "-Xmx2g -Xms2g"
      discovery.zen.ping.unicast.hosts: esclient
    cap_add:
      - IPC_LOCK
    networks:
      - elk
    depends_on:
      - esmaster

  esdata:
    build: elasticsearch/
    volumes:
      - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./elasticsearch/data:/usr/share/elasticsearch/data
    environment:
      cluster.name: "escluster"
      node.data: "true"
      http.enabled: "false"
      node.master: "false"
      ES_JAVA_OPTS: "-Xmx2g -Xms2g"
      discovery.zen.ping.unicast.hosts: esdata
    cap_add:
      - IPC_LOCK
    networks:
      - elk

  logstash:
    build: logstash/
    volumes:
      - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml
      - ./logstash/pipeline:/usr/share/logstash/pipeline
    ports:
      - "5000:5000"
    environment:
      LS_JAVA_OPTS: "-Xmx6g -Xms6g"
    networks:
      - elk
    depends_on:
      - esmaster

  kibana:
    build: kibana/
    volumes:
      - ./kibana/config/:/usr/share/kibana/config
    ports:
      - "5601:5601"
    networks:
      - elk
    depends_on:
      - esmaster

networks:

  elk:
    driver: bridge

Upvotes: 0

Views: 5362

Answers (1)

xeraa
xeraa

Reputation: 10859

You don't want to share your data directory. Each Elasticsearch instance should use it's own data directory.

You can find a working docker-compose file in the section of https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#docker-cli-run-prod-mode — two nodes, each using their own Docker named volume.

Upvotes: 1

Related Questions