CODEWITHSUNDEEP
javaoffsetapache-kafkaproducer-consumerkafka-consumer-api
usman
usman

Reputation: 1391

how to access and commit offsets on __consumer_offsets topic of kafka?

As per Kafka docs

When the offset manager receives an OffsetCommitRequest, it appends the request to a special compacted Kafka topic named __consumer_offsets

is it possible we can access it directly programmatically? Using Kafka Tool 1.0 Beta 2, i can see __consumer_offets topic and lot of partitions in it with only one partition having data. Can not understand it how partitions are created in it and why only one partition have data.

Now also how can i commit on a particular partition in this topic?

Upvotes: 4

Views: 3492

Answers (1)

Edenhill
Edenhill

Reputation: 3113

The layout and contents of the __consumer_offsets topics is a Kafka implementation detail and you should not try to access it directly. Instead use the ConsumerMetadata, OffsetCommit and OffsetFetch API requests described here: https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommit/FetchAPI

Upvotes: 4

Related Questions