CODEWITHSUNDEEP
apache-kafkaapache-zookeeperkafka-consumer-api
Goutam Chowdhury
Goutam Chowdhury

Reputation: 291

Not able fetch offset info for kafka consumergroup using zookeeper CLI

I am using Zookeeper

I am using zookeeper CLI to fetch Offset information for a given consumer group.

Below is working fine 

Command: get /consumers/[ConsumerGroup]/offsets/[TopicName]

o/p : 
0:1640002
cZxid = 0x304ca036d
ctime = Fri Dec 23 17:29:30 UTC 2016
mZxid = 0x30af96e80
mtime = Fri Feb 24 19:15:00 UTC 2017
pZxid = 0x304ca036d
cversion = 0
dataVersion = 113179
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0

But below one is not working 

command: get /consumers/[Same_Consumer_Group]/offsets/[Same_Topic_Name]/0

o/p:Node does not exist: /consumers/[Same_Consumer_Group]/offsets/[Same_Topic_Name]/0

So when I am trying to get offset info using PARTITION_ID then its not working.

Note First Command showing 0:1640002 so it means offset is 1640002 in partition 0. Please help me to solve this problem

Upvotes: 0

Views: 1344

Answers (1)

oh54
oh54

Reputation: 498

Zookeeper is essentially a tree and the numChildren = 0 says that 

/consumers/[ConsumerGroup]/offsets/[TopicName]

is a leaf node and thus as expected when you try to get it's (nonexisting) child node you get an error.

According to https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper from Kafka 0.8 the way offsets are stored in Zookeeper is that 

/consumers/[ConsumerGroup]/offsets/[TopicName]

doesn't contain any data itself and instead the offsets can be found in child nodes 

/consumers/[ConsumerGroup]/offsets/[TopicName]/[Partition]

However, since that is obviously not the case for you what I assume is going on is that you're either using some older version of Kafka or that (for some version of Kafka) for a single partition topic the additional level in the tree isn't created.

But all that shouldn't really concern you anyway in my opinion. Looking at the output you posted it seems you have a topic with 1 partition and the offset for that particular consumer group within that partition is 1640002. That's all the offset info you need, no?

Now if the topic has more than 1 partition then something weird might be going on.

If you think you still have an issue, post your Kafka and Zookeeper versions as well as the number of partitions in the topic in a comment.

Let me know if this helps.

Upvotes: 1

Related Questions