CODEWITHSUNDEEP
javaapache-flink
gibo
gibo

Reputation: 537

Flink stream order

Does Flink guarantee the order of execution of a stream?

I have two Kafka topics (each with single partition), stream 1 and stream 2, and use keyBy, the streams are processed by a coprocess function.

During my testing I can see the the contents of both streams are not always executed in order.

I can set parallelism to 1 to get around this issue, but wanted to understand if what I am seeing is what is expected?

source1.keyBy(e -> e.f0);
source2.keyBy(e -> {
    return new JSONObject(e.f1).getString("someOtherKey");
});

source1.connect(source2)
       .process(new KeyedCoProcessFunction())
       .uid("uid")
       .name("name");

Upvotes: 0

Views: 848

Answers (1)

David Anderson
David Anderson

Reputation: 43707

Events will maintain their relative ordering if they take the same path through the execution graph. Otherwise, all bets are off.

Here's an example of what can happen with a keyBy. Initally the events are ordered as 1, 2, A

enter image description here

but a keyBy sends A down a different path than 1 and 2:

enter image description here

after which we might see A, 1, 2 (as shown here), or 1, A, 2, or 1, 2, A.

enter image description here

Upvotes: 5

Related Questions