CODEWITHSUNDEEP
apache-kafkaapache-zookeeperleader-election
frodo
frodo

Reputation: 1571

Kafka leader election Zookeeper failure

The controller elects a new leader from the ISR for a partition when the current one dies. My understanding is that this data is persisted in Zookeeper. What happens when a Zookeeper node dies during this write? Can this mean that some brokers might still different leader for the newly-leader-elected partition?

I tried digging around the docs but could not find anything satisfactory.

Upvotes: 0

Views: 1201

Answers (1)

JavaTechnical
JavaTechnical

Reputation: 9357

My understanding is that this data is persisted in Zookeeper

Yes, This ISR set is persisted to ZooKeeper whenever it changes. (Reference)

What happens when a Zookeeper node dies during this write?

Zookeeper works on quorum and it means majority of servers from the cluster. (See this SO answer) (snippet below)

  1. With a 3 node cluster, the majority is 2 nodes. So you can tolerate only 1 node not being in sync at the same time.

  2. With a 5 node cluster, the majority is 3 nodes. So you can tolerate only 2 nodes not being in sync at the same time.

So long as there is majority, the decision is made and so the leader election will continue.

Upvotes: 1

Related Questions