CODEWITHSUNDEEP
apache-kafka
B.Y.
B.Y.

Reputation: 43

Kafka Ordering the Messages in Multiple Partitions

I have been trying to print my messages in order with multiple partitions. I know messages are ordered within a single partition but how can I achieve this so that messages are also ordered with multiple partitions?

For example, I send 1,2,3,4,5 and they are sent to different partitions but since they are sent in order from 1 to 5, I also want them to be printed as this order, however they are printed as 2,3,5,4,1. I also have their timestamps but couldn't also implement to order according to this.

Upvotes: 0

Views: 484

Answers (1)

Sanhaji Omar
Sanhaji Omar

Reputation: 332

You can't configure kafka to maintain ordering accross partitions:

you would need to implement your own ordering algorithm according to the timestamp if you want to.

If I'd be you, I'd use kafka as my ingestion entrypoint and I'd store them in another sink to work with the data.

Ordering of events in distributed system is not a simple task:

Upvotes: 1

Related Questions