CODEWITHSUNDEEP
apache-flink
Jerome Li
Jerome Li

Reputation: 176

Flink checkpointing state for non-keyed stream

I am new to Flink. I am trying to enable checkpointing and stateful in my application. I saw how we store keyed state from the Flink documents. But I am wondering can we store non-keyed state (state for ProcessFunction)

Upvotes: 0

Views: 515

Answers (1)

David Anderson
David Anderson

Reputation: 43454

It's somewhat unusual to need non-keyed state, but there is documentation with examples.

In user code this is generally only needed for implementing custom sources and sinks, which is why the examples focus on those use cases. But in a ProcessFunction you would do the same, i.e., implement the CheckpointedFunction interface (i.e., the initializeState and snapshotState methods).

The only types of non-keyed state are ListState, UnionState, and BroadcastState, and ListState is probably the type you want to use. UnionState is very similar to ListState, it just uses a different strategy for redistributing state during rescaling (each parallel instance gets the entire list, instead of being assigned a slice of the list, and the instances are responsible for knowing what to do). BroadcastState is what's used by a BroadcastProcessFunction or KeyedBroadcastProcessFunction.

Upvotes: 1

Related Questions