CODEWITHSUNDEEP
apache-kafkakafka-producer-api
Ankit Sahay
Ankit Sahay

Reputation: 2053

Can sending message to Kafka asynchronously effect message ordering?

This is my understanding of sending messages to Kafka asynchronously and implementing a callback function is:

Scenario: Producer is going to receive 4 batches of messages to send (to same partition, for the sake of simplicity).

Producer sends batch A.

Producer sends batch B.

Producers receives reply from broker and implements call back - batch A was unsuccessful and retriable, batch B was successful, so producer sends batch A again.

Won't this disturb the message ordering as now A is received by Kafka after B?

Upvotes: 1

Views: 461

Answers (1)

Katya Gorshkova
Katya Gorshkova

Reputation: 1561

If you need message ordering within partition you can use idempotent producer:

enable.idempotence=true
akcs=all
max.in.flight.requests.per.connection<=5 
retries>0

This will resolve potential duplicates from producer and maintain the ordering.

If you don't want to use idempotent producer then it is enough to set max.in.flight.requests.per.connection=1. This is the number of unacknowledged batches on the producer side. It means that batch B will not be sent before acknowledge for A is received.

Upvotes: 0

Related Questions