CODEWITHSUNDEEP
apache-kafka
Satya
Satya

Reputation: 636

Consumer group has no active members - Kafka

What is the meaning of the message "Consumer group has no active members" when one tries to describe a kafka-consumer-group?

In-depth explanation appreciated.

Upvotes: 7

Views: 20131

Answers (4)

best wishes
best wishes

Reputation: 6644

The consumer detail may not be present if

a. the consumer was never live

b. the ttl for __consumer_offset is less than the last active time for consumer group.

c. the replication factor for __consumer_offset is 1 and there was data loss for the particular partition.

Upvotes: 0

Joy
Joy

Reputation: 77

One workaround I found is if you can somehow keep your consumer app running (using say while(true) ro something like that) it works.

while (true) {
  ConsumerRecords<String, String> records = consumer.poll(
    Duration.ofMillis(500)
  );

  for (ConsumerRecord<String, String> record : records) {
    logger.info("Key : " + record.key() + " value : " + record.value());
    logger.info(
      "Partition : " + record.partition() + " offset : " + record.offset()
    );
    logger.info("Topic : " + record.topic());
  }
  // break;
}

and then if I run the command in terminal : ./bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my_first_con_1

I get this o/p :

GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID my_first_con_1 my-first-topic 0 9 9 0 consumer-my_first_con_1-1-4dbc5583-e560-4234-81a7-9bd9c9803374 /192.168.1.5 consumer-my_first_con_1-1 my_first_con_1 my-first-topic 1 6 6 0 consumer-my_first_con_1-1-4dbc5583-e560-4234-81a7-9bd9c9803374 /192.168.1.5 consumer-my_first_con_1-1 my_first_con_1 my-first-topic 2 15 15 0 consumer-my_first_con_1-1-4dbc5583-e560-4234-81a7-9bd9c9803374 /192.168.1.5 consumer-my_first_con_1-1

Upvotes: 0

Francisco Albert
Francisco Albert

Reputation: 1721

Sometimes the consumer-group might work but still, when describing the consumer-group you might have "no active members". The reason can be found in here: https://issues.apache.org/jira/browse/STORM-3067

Upvotes: 2

Ryuzaki L
Ryuzaki L

Reputation: 40088

Managing Consumer Groups Yes, members command will only show the consumers that are active in group

> bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group --members

--members: This option provides the list of all active members in the consumer group

What is a Consumer ?

Consumer is a thread that subscribe to one or more topics and process the stream of records

What is Consumer group ?

Kafka uses the concept of consumer groups to allow a pool of processes to divide the work of consuming and processing records. These processes can either be running on the same machine or they can be distributed over many machines to provide scalability and fault tolerance for processing. All consumer instances sharing the same group.id will be part of the same consumer group.

Consumer group has no active members

If consumer threads of specific group are actively polling data from kafka topic are said to be active. If none of consumer threads are active then that group has no active consumers

Upvotes: 5

Related Questions