CODEWITHSUNDEEP
apache-kafkakafka-consumer-api
return 0
return 0

Reputation: 4366

In Kafka, how to replay kafka consumers between two offsets?

Given two offsets - a start and end offset, or start/end datetime timestamp (equally fine), I want a Kafka consumer to replay all messages within that window.

I have figured out how to reset offset using kakfa-consumer-groups.sh tool to reset the offset based on datetime or offset, but how do I tell the consumer to stop after say replaying for 10,000 messages or 10 minutes?

Upvotes: 2

Views: 2379

Answers (1)

Michael Heil
Michael Heil

Reputation: 18475

There is no configuration or API that lets you stop a KafkaConsumer after a certain amount of processed offsets or time.

You would need to do this programatically by checking the offsets of the ConsumerRecord or having a timer that will stop the Consumer after a certain time.

Instead of using kafka-consumer-groups tool you could also make use of the seek API of the KafkaConsumer to start at a particular offsets for a partition.

Upvotes: 2

Related Questions