CODEWITHSUNDEEP
elasticsearchelasticsearch-5
Menelaos
Menelaos

Reputation: 26004

Elastic Search Cluster Config - automatic discovery , config, and multi-elastic on one physical

Questions

So we are using elasticsearch-5.6.12 and I was wondering:

  1. Is there was a way to enable the automatic discovery instead of having to add all the IP addresses within the configuration file?
  2. Any issues with the below configuration?
  3. Would there be any problems with running 2 elastic processes on the same physical machine?

Configuration

Currently I have the following config on the first server:

cluster.name: mbak-prod-elastic-search
node.name: mbak-PROD1.net
network.host: 192.168.212.191
http.port: 8200
discovery.zen.ping.unicast.hosts: ["192.168.212.191", "192.168.212.192", "192.168.212.193"]
discovery.zen.minimum_master_nodes: 2
node.data: true
node.master: true

I also have the following on two other servers:

   cluster.name: mbak-prod-elastic-search
    node.name: mbak-PROD2.net
    network.host: 192.168.212.192
    http.port: 8200
    discovery.zen.ping.unicast.hosts: ["192.168.212.191", "192.168.212.192", "192.168.212.193"]
    discovery.zen.minimum_master_nodes: 2
    node.data: true
    node.master: true

AND

  cluster.name: mbak-prod-elastic-search
    node.name: mbak-PROD3.net
    network.host: 192.168.212.193
    http.port: 8200
    discovery.zen.ping.unicast.hosts: ["192.168.212.191", "192.168.212.192", "192.168.212.193"]
    discovery.zen.minimum_master_nodes: 2
    node.data: true
    node.master: true

Additionally, is there anything here that doesn't look correct or could cause problems?

I did a test and saw that the cluster continues to run even if one of the nodes is down. There need to be atleast two nodes running at any one time.

Update

I am realizing, by reading the documentation more, that we should maybe have our master-eligable nodes be separate then the data nodes? Are there any major problems with having our data nodes also be master-eligable?

Is it necessary to move the master eligibility off the data nodes?

I was watching and reading the following:

Upvotes: 0

Views: 356

Answers (2)

hamid bayat
hamid bayat

Reputation: 2179

It is not recommended to run multiple instance on one pyshical machine. but we done it because of we have very large RAM capacity. (I dont know why you want to do this)

the correct configuration:

cluster.name: mbak-prod-elastic-search node.name: mbak-PROD1.net
network.host: 192.168.212.191 
transport.tcp.port: 9300 
http.port: 8200 
discovery.zen.ping.unicast.hosts: ["192.168.212.191", "192.168.212.192:9302", "192.168.212.193:9304"] 
discovery.zen.minimum_master_nodes: 3 
node.data: true 
node.master: true



cluster.name: mbak-prod-elastic-search
node.name: mbak-PROD2.net
network.host: 192.168.212.192
transport.tcp.port: 9302
http.port: 8202
discovery.zen.ping.unicast.hosts: ["192.168.212.191:9300", "192.168.212.192", "192.168.212.193:9304"]
discovery.zen.minimum_master_nodes: 3
node.data: true
node.master: true



cluster.name: mbak-prod-elastic-search
node.name: mbak-PROD3.net
network.host: 192.168.212.193
transport.tcp.port: 9304
http.port: 8204
discovery.zen.ping.unicast.hosts: ["192.168.212.191:9300", "192.168.212.192:9302", "192.168.212.193"]
discovery.zen.minimum_master_nodes: 3
node.data: true
node.master: true

Upvotes: 0

Ijaz Ahmad
Ijaz Ahmad

Reputation: 12110

You can run master and data role on the same node , but is not recommended

You need to configure the minimum master nodes according to the formula, to avoid split brain, for example , if you have 3 master nodes , then you need min : 2

A quorum is (number of master-eligible nodes / 2) + 1

For discovery , I think you just need to put the master nodes in the list, will work

Your config seems ok but you should run master and data roles on separate nodes , in large scale production environment.

Note:

Apart from Zen discovery , you can use the following in cloud environments:

  • EC2 discoery
  • GCP discovery
  • Azure discovery

Upvotes: 1

Related Questions