CODEWITHSUNDEEP
apache-kafkakafka-consumer-apiapache-kafka-streams
George Smith
George Smith

Reputation: 1067

Delete unused kafka consumer group

I'm using Apache Kafka 0.10 with a compacted topic as a distributed cache synch mechanism. When the application starts up it generates an instance specific consumer group id. As instances are added and removed for horizontal scalability, obviously we get a large number of group ids that should never be used again.

I'm sure that this is the perfect use case for KStreams and KTables, but I am trying to do this myself for intellectual reasons as well as that the KStreams and KTables are defined as alpha quality in 0.10.

Is there a Kafka API call that I can use that could delete an existing consumer group, knowing that it should never be used again?

Since Zookeeper is not maintaining consumer offsets in version 0.10, Is there a way delete the consumer group using Kafka?

Upvotes: 9

Views: 14154

Answers (2)

mrsrinivas
mrsrinivas

Reputation: 35434

It's possible with CLI in Kafka

./bin/kafka-consumer-groups \
    --bootstrap-server <bootstrap_server(s)> \
    --topic <topic_name> \
    --delete \
    --group <consumer_group_name>

kafka-consumer-groups should be available in Kafka installation home.

Upvotes: 7

Matthias J. Sax
Matthias J. Sax

Reputation: 62350

Since Kafka 0.9, an internal topic is used to store committed offsets. You can configure how long those offsets should be kept via offsets.retention.minutes. (See also offsets.retention.check.interval.ms).

Upvotes: 5

Related Questions