CODEWITHSUNDEEP
apache-kafkaapache-zookeeper
khg
khg

Reputation: 39

Zookeeper cannot work correctly if one node down in cluster

I had problem on my cluster zookeeper, i have 2 server and each server have one zookeeper. ex : 

server.1=x.x.x.x:2888:3888 
server.2=x.x.x.x:2888:3888

if one zookeeper down, the other one zookeeper can't run. I don't have any idea to solve this problem, or should the best number zookeeper i have to build cluster zookeeper ?

Upvotes: 2

Views: 3460

Answers (2)

Giorgos Myrianthous
Giorgos Myrianthous

Reputation: 39930

In a highly available Zookeeper ensemble you must configure 2n + 1 Zookeeper instances where n is any number greater than 1. This means that for a Zookeeper quorum (i.e. a healthy ensemble), you must configure an odd number of instances (3, 5, 7, ...). This is because Zookeeper cannot perform majority elections for leadership using an even number of instances.

Assuming that you have correctly configured a Zookeeper ensemble with 2n + 1 instances (quorum), there can be up to n failed Zookeeper instances without taking the cluster down.

If the quorum breaks, then the Zookeeper cluster will go down. This is what had happened in your case. Try to use a higher number of Zookeeper instances in your ensemble (3 or 5) and it should do the trick. Alternatively, if you don't need high availability you can still use just 1 Zookeeper instance.

Upvotes: 2

Mark Bramnik
Mark Bramnik

Reputation: 42541

ZooKeeper will work as long as majority of its nodes if up, if N is number of nodes in cluster it will work as long as working_nodes_number > N/2

Now if you have 2 nodes, and one is down (working_nodes_number = 1), zoo keeper won't work: N / 2 = 1 and obviously 1 > 1 is wrong

In other words, cluster of two servers can't stay alive even if one node is down.

Try running 3 nodes, so that if one node goes down, there still be 2 nodes (majority) so that zookeeper will keep working

In general its a good strategy too use cluster of an odd number of nodes. You might be interested to read this article that is relevant to the discussion

Upvotes: 1

Related Questions