CODEWITHSUNDEEP
apache-flinkflink-streaming
YuFeng Shen
YuFeng Shen

Reputation: 1565

Why "broadcast state" can store the dynamic rules however broadcast() operator cannot?

I got confused for the difference between "broadcast state" and broadcast() operator, and finally I got the help from a Flink expert in the following thread.

What does it mean that "broadcast state" unblocks the implementation of the “dynamic patterns” feature for Flink’s CEP library?

In the end it seems got the conclusion that "broadcast state" can store the dynamic rules in the keyed stream by RichCoFlatMap , however broadcast() operator cannot, so may I know how "broadcast state" store the dynamic rules by RichCoFlatMap and why broadcast() operator cannot store the dynamic rules by RichCoFlatMap? May I got am example for explaining it?

Upvotes: 0

Views: 601

Answers (2)

lafeier
lafeier

Reputation: 11

They are different concepts, BroadcastState is a storage concept, and Broadcast () is an operation whose purpose is to build BroadcastStream for you.

Upvotes: 0

Dawid Wysakowicz
Dawid Wysakowicz

Reputation: 3422

Those are completely two different concepts. Moreover the broadcast() is kind of a prerequisite for BroadcastState.

broadcast() specifies partitioning of data, that says that each element of the stream should be broadcasted to each parallel downstream operator.

BroadcastState is a state of operator that first of all allows to be read-write from a broadcasted stream and read from non-broadcasted one. Before that there was no way to join such two streams. Moreover this state will ensure that after restore each instance of the state across all parallel instances will be the same.

For more information on the BroadcastState have look into this docs.

Upvotes: 1

Related Questions