CODEWITHSUNDEEP
apache-kafkapython-3.7kafka-python
user11322908
user11322908

Reputation: 23

Find the topics under a consumer group in Kafka using Kafka-Python

Input to my python script is the name of a consumer group and the output is supposed to be the list of topics under the consumer group.
My code currently, doesn't return the topics for which the current offset is -1.
Is there a better way, by which I can obtain the list of all topics under the consumer group using Kafka-python.
I am actually looking to replace describe a topic in a group cmd using Kafka tools.

from kafka import KafkaAdminClient
topics_in_groups = {};
client = KafkaAdminClient(bootstrap_server='localhost:9092')
for group in client.list_consumer_groups():
  topics_in_groups[group[0]] = [];
  topic_dict = client.list_consumer_group_offsets(group[0]);
  for topic in topic_dict:
    topics_in_groups[group[0]].append(topic.topic)

Upvotes: 0

Views: 1902

Answers (2)

Dheepan Ram
Dheepan Ram

Reputation: 11

I tried this and worked.

from kafka import KafkaAdminClient
topics_in_groups = {}
client = KafkaAdminClient(bootstrap_servers=['broker:9092'])

for group in client.list_consumer_groups():
    topics_in_groups[group[0]] = []

for group in topics_in_groups.keys():
    my_topics = []
    topic_dict = client.list_consumer_group_offsets(group)
    for topic in topic_dict:
        my_topics.append(topic.topic)
        topics_in_groups[group] = list(set(my_topics))

for key , value in topics_in_groups.items():
    print(key, "\n\t", value)

Upvotes: 1

Niranjan
Niranjan

Reputation: 157

client.list_consumer_group_offsets(group[0]) will get you the metadata of consumer group, the below code will give you the list of topics and partitions of a group.

topics_in_groups = []
for topic,prtitions in topic_dict:
    topics_in_groups.append({"topic":topic, "partition": partition})

Upvotes: 1

Related Questions