Tom
Tom

Reputation: 1

How to process kafka batch events in parallel with order guarantees for duplicate IDs in Apache Camel?

I have a problem with parallel processing in Apache Camel. I am consuming a batch of 10 messages from Kafka, which gives me an exchange with list of exchanges. I want to process these exchanges in parallel.

The issue arises when two or more events in the batch refer to the same object, e.g., an order that is updated twice. In such cases, the batch contains two events with the same ID. For these events, I need to process them in the exact order they were received in the batch, while still processing the other 8 events in parallel.

I was trying to use aggregation to check if an element repeats and, if so, move it to a different route for sequential processing, while allowing others to be processed in parallel, but I wonder if Apache Camel natively support any solutions that would make it easier to implement?

Upvotes: 0

Views: 18

Answers (0)

Related Questions