CODEWITHSUNDEEP
apache-flinkflink-streaming
alex
alex

Reputation: 31

flink aggregation with window AND state

I'm looking for a way to implement aggregation/fold function on a window that also have a state. I understand how to aggregate on a window, and how to use key/global state - but not both.

Just to be clear, when I say a window with state - I mean that the state should be initialized (nullified) every time the window is changed/moved.

For example: I want to count the number of events keyed by event type every 5 minutes. But in addition to event type (which is the window key) the event has some id field - and I would like to count each id only once - so I need to save a state of all the ids I've already counted in that window.

Is there a simple way to do this in Flink?

Upvotes: 3

Views: 1975

Answers (1)

David Anderson
David Anderson

Reputation: 43717

Flink has a RichReduceFunction, which will give you access to state that is global across all windows for a given key. If you need per-window state, see [Flink-5929] which will be part of Flink 1.3.

Upvotes: 2

Related Questions