CODEWITHSUNDEEP
apache-kafkakafka-consumer-api
Erik Pettersson
Erik Pettersson

Reputation: 479

Kafka auto commit interval best practice

Are there any best practices when selecting value of auto.commit.interval.ms?

I read here that:

In general, it is not recommended to keep this interval too small because it vastly increases the read/write rates in zookeeper and zookeeper gets slowed down because it's strongly consistent across its quorum.

What is too small? Is this still an issue with kafka >= 0.9.0 version?

Upvotes: 2

Views: 3180

Answers (1)

Harald
Harald

Reputation: 5093

The question is not what is small, but rather what can you live with? If you can live with re-processing several minutes of messages in case of a crash of your consumer, you could set the interval to a few minutes. Because this is what it is all about: messages will be processed after a (re)start from the last committed offset.

Upvotes: 5

Related Questions