CODEWITHSUNDEEP
javaapache-flink
ethrbunny
ethrbunny

Reputation: 10469

flink: applying multiple aggregations on a windowed stream

I have some data coming in as id, float, float, float. I want to min(), max() and sum() the fields in order and group this by the id value.

Using flatMap I have a Tuple4 with the bits but I'm not sure how to send it to the next step.

What I have:

dataStream.flatMap(new mapper()).keyBy(0)  
   .timeWindowAll(Time.of(5, TimeUnit.SECONDS)).min(1)  
   .timeWindowAll(Time.of(5, TimeUnit.SECONDS)).sum(2)
   .timeWindowAll(Time.of(5, TimeUnit.SECONDS)).sum(3)
   .map(new printstuff());

Is this the correct way to handle this? Or do I need to put each timeWindowAll in its own statement with keyBy and so forth?

Upvotes: 5

Views: 2275

Answers (1)

Fabian Hueske
Fabian Hueske

Reputation: 18987

Chaining of multiple aggregation functions is not supported in the DataStream API yet.

In your example, you create three distinct 5-second windows each of which applies a single aggregation. This is probably not what you want to do. I would implement a custom ReduceFunction that performs all aggregations at once in a single window. See Window Reduce in the DataStream documentation for an example.

Upvotes: 7

Related Questions