CODEWITHSUNDEEP
apache-kafkaapache-zookeeper
sam
sam

Reputation: 3511

Kafka describe cmd always return at least one alive partition

I am just now starting to learn about kafka and I am following the introduction part of the docs. As I understand it the command "describe" should indicate how many replicas of a partition are alive and well in the cluster. From the docs

"isr" is the set of "in-sync" replicas. This is the subset of the replicas list that is currently alive and caught-up to the leader.

But after shutting down all my nodes (zookeeper still running) and running describe :

$ bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic test-replicated

I get the following result : seeming to indicate that nbroker 2 has a partition alive and well ?

Topic:test-replicated   PartitionCount:1        ReplicationFactor:3     Configs:
    Topic: test-replicated  Partition: 0    Leader: 2       Replicas: 1,2,0 Isr: 2

Could someone explain to me why ? Maybe I am misunderstanding something about kafka ?

EDIT

Just noticed that after restarting only my node 0 the output of the same cmd now indicates that broker "0" is the owner of a partition alive and working. Does that mean that when there is no node alive the broker displayed is the last one that was running before all went down ?

Topic:test-replicated   PartitionCount:1        ReplicationFactor:3     Configs:
    Topic: test-replicated  Partition: 0    Leader: 0       Replicas: 1,2,0 Isr: 0

Upvotes: 0

Views: 307

Answers (1)

Nipun Talukdar
Nipun Talukdar

Reputation: 5387

The data for topic details are coming from Zookeeper and for this connecting to Kafka brokers is not necessary. The information for a particular partition is generally stored in a znode (/brokers/topics/topicname/partitions/partition-number). They are updated by the controller in the Kafka cluster. Generally, the Kafka server which is started first becomes the controller (until it goes down or lose contact to Zookeeper). When you shutdown Kafka cluster, all the topic details will still remain in Zookeeper, and that won't be updated by anyone as the controller node also went down. Hence, you will continue to see the "stale" topic-partition details from kafka-topics.sh command. As soon as a server starts again, it becomes controller and it will detect which nodes are up and which are down and it will update information in zookeeper.

Upvotes: 2

Related Questions