How to get the offsets programmatically with Kafka 1.0

Question

The OffsetRequest has been deprecated for a while, along with almost all the other classes in kafka.api and now has been removed in 1.0. However, neither the deprecation message nor the docs explain what can be used instead.

The FAQ is not helpful for this topic.

There are CLI tools for this, but I have not found any advice on doing it programmatically.

Answer 1

The classes in kafka.api are for the old clients (in Scala) that are deprecated and will be removed in a future release.

The new Java clients are using the classes defined in org.apache.kafka.common.requests.

The class org.apache.kafka.common.requests.ListOffsetRequest is the replacement for the old OffsetRequest.

The following methods from KafkaConsumer can be used to retrieve offsets (they all send a ListOffsetRequest from the client):

• beginningOffsets() : http://kafka.apache.org/10/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#beginningOffsets-java.util.Collection-

• endOffsets(): http://kafka.apache.org/10/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#endOffsets-java.util.Collection-

• offsetsForTimes(): http://kafka.apache.org/10/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#offsetsForTimes-java.util.Map-