CODEWITHSUNDEEP
apache-kafka
dimamah
dimamah

Reputation: 2903

How to check if a topic was consumed by a consumer in Kafka

How should one check if a specific consumer group, fully consumed a topic?
This is the equivalent of checking if a queue is empty in standard queuing system.
This isn't obvious as each consumer only see the partitions it is given hence it isn't aware if other consumers in it's group consumed their part.
I guess this should be done somehow by AdminUtils or ZkUtils but I cant seem to find the right way.

Upvotes: 6

Views: 8606

Answers (1)

Shawn Guo
Shawn Guo

Reputation: 3228

Please use kafka-consumer-groups.sh to check consumer offset.
(The kafka-consumer-offset-checker.sh has similar functionality but has been deprecated in 0.9.0.0.)

bin/kafka-consumer-groups.sh --bootstrap-server xxx:9092--group xxx-group-id --describe --new-consumer
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG OWNER consumer.test.0529.095217 secured_topic 0 586 3333 2747 consumer-1_/10.139.0.4 consumer.test.0529.095217 secured_topic 3334 3334 0 consumer-1_/10.139.0.4 consumer.test.0529.095217 secured_topic 2 0 3333 3333 consumer-1_/10.139.0.4

Upvotes: 9

Related Questions