Apache Kafka 0.9 Java API Consuming all messages from beginning of topic

Question

I need to be able to consume all messages from a topic, from the beginning. Essentially identical to this StackOverflow query, but updated for Kafka 0.9. (There seem to be relatively few 0.9-specific StackOverflow answers).

Kafka High Level Consumer Fetch All Messages From Topic Using Java API (Equivalent to --from-beginning)

0.9 has a sufficiently different API that I don't really know where to start. I can do this from the command-line using a provided bash script, but don't know how to move forward.

Could you please provide me the appropriate methods or a small sample script to get me started? Thank you!

Answer 1

You need to set auto.offset.reset to earliest. See https://kafka.apache.org/documentation.html#newconsumerconfigs

See also https://github.com/apache/kafka/blob/0.9.0/core/src/main/scala/kafka/tools/ConsoleConsumer.scala#L179