CODEWITHSUNDEEP
apache-flinkflink-streaming
Sergio Perea
Sergio Perea

Reputation: 21

How to execute functions on empty windows in Flink streaming?

I wrote a Flink program that calculates the number of events per keyed window from a simple kafka stream. I works great, fast & accurate. When the source stops, I would like to have 0 as result of the calculation on each window, but no result is sent. The function just does not execute. I assume this is because of the lazy operation behavior of Flink.

Any recommendation?

Upvotes: 2

Views: 1223

Answers (1)

Hadrien Lepousé
Hadrien Lepousé

Reputation: 141

I encountered the same situation. Filling the holes in your database with another process is a solution.

However, I found it easier to union your main stream with a custom periodical source, that emits dummies, whose only roles are to trigger windows creation. When doing this, you have to make sure that dummies are ignored in computations.

Here is how to code a periodical source (however you may not need a RichParallelSourceFunction, a SourceFunction can be enough)

Upvotes: 2

Related Questions