Maintaining FIFO order of Kafka topic partitions in OpenWhisk

Question

I am looking for FIFO order maintenance when I bind Kafka Topic with OpenWhisk. I could not find any clear documentation on this specific topic. Following link describes that Actions are not ordered (https://www.oreilly.com/library/view/learning-apache-openwhisk/9781492046158/ch01.html). However, I need to follow strict ordering in my scenario as described below:

• Messages for Object 1: [M11 M12 M13]
• Messages for Object 2: [M21 M22 M23]

I am going to use Kafka Topic Partitions to send messages belonging to same object in a sequential manner, as in my scenario I am concerned about message ordering for a single object.

Now when we consume these messages in OpenWhisk how does it ensure that messages for an object are handled in order e.g., if M11 is currently being processed then does it also start processing M12 without waiting for M11 to complete?

I am new to OpenWhisk and currently doing research on it to understand that it caters to my use cases.

Answer 1

I think this is more of a Kafka question than an OpenWhisk.

How a kafka queue is processed is defined by the number of partitions you have. You will have FIFO behaviour within the single partition, but if you have more, which I belive is default, it is non-deterministic.

So the solution would be to have only one partition in Kafka.

Also for true FIFO behaviour, you need to have client start reading from the first message in the partition.