CODEWITHSUNDEEP
javaapache-kafkakafka-producer-api
pavel_barbashov
pavel_barbashov

Reputation: 122

Kafka message ordering in partition while producer retry

According to producer configs, there are: retries and max.in.flight.requests.per.connection. Suppose that retries > 0 and max.in.flight.requests.per.connection > 1.

Can messages arrive out of order within ONE partition of topic (e.g. if first message has retries, but second message delivered to broker with the first attempt)?

Or do out of order only happen across several partitions of topic, but within partition order is preserved?

Upvotes: 4

Views: 1524

Answers (2)

Ak G
Ak G

Reputation: 43

As per latest update documentation, you can have maximum 5 max.in.flight.requests.per.connection and Kafka can maintain order for this.

Upvotes: 1

Mickael Maison
Mickael Maison

Reputation: 26865

If you set retries to more than 0 and max.in.flight.requests.per.connection to more than 1, then yes messages can arrive out of order on the broker even if they are for the same partition.

You can also have duplicates if for example a message is correctly added to the Kafka logs and an error happens when sending the response back to the client.

Since Kafka 0.11, you can use the Idempotent producer to solve these 2 issues. See http://kafka.apache.org/documentation/#semantics

Upvotes: 7

Related Questions