CODEWITHSUNDEEP
asynchronousapache-flinkdatasource
Olgun Kaya
Olgun Kaya

Reputation: 2579

Apache Flink Rules over Async Data Streams

I have been struggling with flink features to be able to solve below detailed problem;

  1. There are more than one data source fed asynchronously. Means; One message source sends messages in every X seconds and other message source sends messages in every Y seconds.

  2. I would like to apply some rules based on the latest values of these sources' specific values; for example;

val(x) > 10 and val(y) > 20

  1. What I have done so far is; Filtering these two different sources and getting the only values I do care about and keeping them in a hash map and feeding another stream with this hash map.

hash map stream last element life cycle

  1. This process event runs the rule over per hash map instance and then take appropriate action.

Upvotes: 0

Views: 138

Answers (1)

kkrugler
kkrugler

Reputation: 9245

If you have two streams, you can connect them and feed them into a CoFlatMapFunction, which can then keep track of the latest value from each stream, and emit results based on the application of your rules to these values.

Upvotes: 1

Related Questions