CODEWITHSUNDEEP
springapache-kafkaspring-kafka
raikumardipak
raikumardipak

Reputation: 1585

How is offset committed in Spring Kafka?

I am using Spring Kafka implementation to integrate with Kafka.

And am struggling to find out how internally Spring Kafka is handling offset commit.

I need this knowhow to decide my strategy on Disaster Recovery while switching from one Kafka Broker to the DR Kafka Broker.

Please help or route me to a post/blog which explains how offset commits are handled by Spring's implementation of Kafka. Thanks.

Upvotes: 0

Views: 291

Answers (1)

Artem Bilan
Artem Bilan

Reputation: 121262

See documentation for some info: https://docs.spring.io/spring-kafka/docs/current/reference/html/#committing-offsets.

In the end the commit is delegated to the KafkaConsumer any way:

this.consumer.commitSync(commits, this.syncCommitTimeout);

or

this.consumer.commitAsync(commits, (offsetsAttempted, exception) -> {

So, when you switch from one broker to another without clustering between them, all those commits and offset tracking does not make sense. Just because the data on a new broker is fully new and it has its own offsets, even if topic name and partitions are the same over there.

Upvotes: 1

Related Questions