CODEWITHSUNDEEP
apache-kafka
Raman
Raman

Reputation: 717

Kafka: Describe Consumer Group Offset

While the Kafka consumer application is up and running, we are able to use the kafka-consumer-groups.sh to describe and retrieve the offset status.

However, if the application goes down, then the command just displays the application is in REBALANCING.

Is there a way to just see the lag of a particular consumer group, even if the application is not up and running?

For example, I would like this output

GROUP|TOPIC|PARTITION|CURRENT-OFFSET|LOG-END-OFFSET|LAG
hrly_ingest_grp|src_hrly|4|63832846|63832846|0
hrly_ingest_grp|src_hrly|2|38372346|38372346|0
hrly_ingest_grp|src_hrly|0|58642250|58642250|0
hrly_ingest_grp|src_hrly|5|96295762|96295762|0
hrly_ingest_grp|src_hrly|3|50602337|50602337|0
hrly_ingest_grp|src_hrly|1|29288993|29288993|0

Upvotes: 0

Views: 2291

Answers (2)

Ryuzaki L
Ryuzaki L

Reputation: 40038

Even the consumer application is down, this command will show the offset of each consumer of that group

bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group

Output:

TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                    HOST            CLIENT-ID
topic3          0          241019          395308          154289          consumer2-e76ea8c3-5d30-4299-9005-47eb41f3d3c4 /127.0.0.1      consumer2
topic2          1          520678          803288          282610          consumer2-e76ea8c3-5d30-4299-9005-47eb41f3d3c4 /127.0.0.1      consumer2
topic3          1          241018          398817          157799          consumer2-e76ea8c3-5d30-4299-9005-47eb41f3d3c4 /127.0.0.1      consumer2
topic1          0          854144          855809          1665            consumer1-3fc8d6f1-581a-4472-bdf3-3515b4aee8c1 /127.0.0.1      consumer1
topic2          0          460537          803290          342753          consumer1-3fc8d6f1-581a-4472-bdf3-3515b4aee8c1 /127.0.0.1      consumer1
topic3          2          243655          398812          155157          consumer4-117fe4d3-c6c1-4178-8ee9-eb4a3954bee0 /127.0.0.1      consumer4

Upvotes: 0

Bartosz Wardziński
Bartosz Wardziński

Reputation: 6593

You can use kt (Kafka tool) - https://github.com/fgeller/kt

Command to query offset and lag will be as follow:

kt group -group groupName -topic topicName -partitions all

Upvotes: 0

Related Questions