CODEWITHSUNDEEP
apache-kafka
edjm
edjm

Reputation: 5522

What is kafka-topics.sh --describe showing me?

Can you explain what kafka-topics.sh --describe is showing? me I am following a tutorial video and also was reading the Apache documentation but I need a little more clarify as to what I'm looking at for the following columns in this graphic.

Leader: Is this pointing to the 3rd broker or is this pointing to the 3rd partition [2]?

Replicas: Is this pointing to brokers:partitions?

Isr: Is this pointing to brokers:partitions?

I would greatly appreciate it if someone explains what the columns A, B , C, D are.description of topic

Upvotes: 4

Views: 5207

Answers (1)

H.Ç.T
H.Ç.T

Reputation: 3569

Topic name: "install_test2"

4 partitions (partition 0, partition 1, partition 2, partition 3) and your replication factor for this topic is 2. It means that data in your topic will be stored (replicated) in 2 brokers for redundancy. In Kafka every partition has a leader and all the requests from producers and consumers are sent to the leader.

Leader column (column B in your image) shows broker ids of the leader for each partition. (Kafka evenly distributes partition leadership between brokers for load balancing)

Replicas column (column C in your image) shows ids of brokers that replicates data for each partition. The first id represents preferred leader. It means Kafka will try to make this broker leader of partition.

ISR (column D in your image) means in-sync-replica. In Kafka when a message is sent to a topic-partition (firstly message is received and stored in leader) and if you have replication factor greater than 1, then replica broker(s) send fetch request and this data is replicated to other broker(s). A follower (replica) broker is in-sync if it is not far behind the leader (explained in below). If a partition leader fails, Kafka chooses an ISR as the new leader for failover.

From Kafka docs:

Configuration parameter replica.lag.time.max.ms now refers not just to the time passed since last fetch request from replica, but also to time since the replica last caught up. Replicas that are still fetching messages from leaders but did not catch up to the latest messages in replica.lag.time.max.ms will be considered out of sync.

Upvotes: 10

Related Questions