CODEWITHSUNDEEP
apache-kafkaspring-batch
One Developer
One Developer

Reputation: 566

Timeout of 60000ms expired before successfully committing the current consumed offsets

For testing purposes, I posted 5k messages on a Kafka topic and I am using a pull method to read 100 messages every iteration in my spring batch application, it runs for ~2hrs before it finishes.

Facing below error at times and execution is getting stopped.

org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before successfully committing offsets

what could be the reason and fix?

Upvotes: 2

Views: 8495

Answers (1)

xuanzjie
xuanzjie

Reputation: 51

Did you consume all the message in two hours? If it is still consuming, MAX_POLL_INTERVAL_MS_CONFIG may be triggered. The default is 5 minutes. If the interval between poll() is more than 5 minutes, it will be kicked out of the consumer group and rebalanced. During this process, the consumer group is unavailable. I am not clear about more information, just provide a solution direction.

fix by 20211012:"If it is still consuming, MAX_POLL_INTERVAL_MS_CONFIG may be triggered" means that if the consumption has not been finished, then the consumption rate is very slow, so this mechanism may be triggered. You can adjust this parameter to try to verify.

Upvotes: 1

Related Questions