CODEWITHSUNDEEP
apache-kafkakafka-consumer-api
Vish
Vish

Reputation: 847

When do Kafka consumer retries happen?

There is a retry feature on Kafka clients. I am struggling to find out when a retry happens. Would a retry happen if the connection to the broker in interrupted briefly? How about if the brokers were not reachable for 5 mins? Will the messages get delivered once the brokers are back up? Or does retry only happen on known scenarios to the kafka clients?

Upvotes: 1

Views: 487

Answers (1)

Ryuzaki L
Ryuzaki L

Reputation: 40078

Kafka Producer consists of a pool of buffer space that holds records that haven't yet been transmitted to the server as well as a background I/O thread that is responsible for turning these batch records into requests and transmitting them to the cluster.

For example if records are sent faster than they can be delivered to the server the producer will block for max.block.ms after which it will throw an exception. Then client assumes batch is failed and will retry to send the batch based on retries config

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for my-test-topic-4 due to 30024 ms has passed since batch creation plus linger time

Suppose if the retries config is set to 3 and if all retries fails then the batch is lost

error: Failed to send message after 3 tries

How about if the brokers were not reachable for 5 mins?

If the broker is down and in mean time if retry is exhausted then you lost the data

Upvotes: 2

Related Questions