CODEWITHSUNDEEP
elasticsearch
Igor Trivunic
Igor Trivunic

Reputation: 55

elasticsearch 2.1.1 - cannot establish cluster

I am having a problem with ES 2.1.1 where every node that I start up, starts as master and I cannot establish an actual cluster. Basically, I end up with 3 clusters of one node each, instead of one 3-node cluster. This is not an issue if I start up ES 1.4.2 with the same configuration files.

Thinking I was suffering a split-brain problem, I started up 3 nodes, and even had 

discovery.zen.minimum_master_nodes: 2

in my config, but it did not make a difference. None of the nodes are aware of any other nodes and no master is ever elected. It does wait for other nodes and doesn't elect itself as master, but it never discovers any other nodes joining in. I'm assuming the pings aren't being sent out at all.

Every node has the same config file, with "node.name" being unique to each node and it all works just fine on ES 1.4.2. Not sure what to do to get it working on ES 2.1.1.

The full config file looks like this:

cluster.name: my_test_cluster
node.name: "my_node1" (diff for each node)
discovery.zen.ping.timeout: 30s
network.host: _non_loopback:ipv4_
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping.multicast.enabled: true

(The last two lines added as an attempt to eliminate split-brain as one of the reasons this wasn't working.)

The logs don't show anything wrong other than a "MasterNotDiscoveredException[waited for [30s]]" when I execute a CURL query to check the status/health of the cluster. Which is expected.

Anyone have any thoughts on why the difference in behavior between the two and how to get 2.1.1. working?

As per suggestion below i have included an entry for node.host in the config files, but the behavior has not changed.

Upvotes: 0

Views: 946

Answers (1)

pickypg
pickypg

Reputation: 22332

You also need to set network.host for each node starting with Elasticsearch 2.0.

I would suggest using either _non_loopback:ipv4_ or _non_loopback:ipv6_. If you're running in AWS, then the discovery-ec2 plugin has separate options that are specific to that environment.

Without setting this, it defaults to only using _local_, which (as the documentation notes) sets it to loopback addresses such as 127.0.0.1, ::1.

Based on your comment, it looks like the nodes are on a different machine, so you also need to specify the list of hosts (a subset of them anyway, but for 3 it's okay to specify all of them):

discovery.zen.ping.unicast.hosts: ["host1", "host2"]

Starting with ES 2.x, multicast is disabled by default, so it will no longer discover other nodes on the network.

Upvotes: 1

Related Questions