CODEWITHSUNDEEP
apache-kafkakafka-consumer-api
SunilS
SunilS

Reputation: 2278

Kafka Consumer Group Metadata

Kafka Version 2.4.1 Zookeeper 3.4.10

I am having trouble deleting a consumer group created by kafka streams application. Even though the application is down, but the group always report it as 'rebalancing' state. And kafka-consumer-groups doesn't allow me to delete it as it says 'The group is not empty'

Refer: Consumer group stuck in 'rebalancing' even though there are no consumers

Questions:

  1. Where is consumer group information stored ? ( I tried zkcli.sh command and /consumers is empty)

  2. Where does group coordinator store information related to consumer & its state

  3. Is there other commands which can be used to query the group information ( or any hack)
  4. Where are consumer group members stored ?

Upvotes: 1

Views: 2298

Answers (1)

Giorgos Myrianthous
Giorgos Myrianthous

Reputation: 39800

In order to list all consumer groups 

bin/kafka-consumer-groups.sh  \
  --bootstrap-server localhost:9092 \
  --list

Now to get information about a specific consumer group you can simply run 

bin/kafka-consumer-groups.sh \
    --bootstrap-server localhost:9092 \
    --describe \
    --group my-consumer-group
  1. Where are consumer group members stored ?

The metadata of consumer groups are stored in internal topic __consumer_offsets.

Upvotes: 1

Related Questions