CODEWITHSUNDEEP
apache-flinkflink-streaming
Rad Rad
Rad Rad

Reputation: 11

Flink change stream window size during run time

I have one stream is consumed by FlinkKafkaConsumer which will be joined with another stream for defined window size such as Time.milliseconds(10000).

How can I change window size during runtime to Time.milliseconds(20000)? 

Stream1.join(Stream2) 
     .where(new SingleValueSensorKeySelector()) 
     .equalTo(new GPSKeySelector()) 
     .window(TumblingEventTimeWindows.of(Time.milliseconds(10000))) 
     .apply(joinStreamFunc).addSink(kafkaProducer);

Upvotes: 1

Views: 781

Answers (1)

David Anderson
David Anderson

Reputation: 43499

The window size can't be changed at runtime.

There are a couple of things you could do, though neither is very attractive. You could implement your own windowing on top of some sort of ProcessFunction. Or you could duplicate the streams involved, have both kinds of windowing going on in parallel, and then dynamically control which window's results go to the sink.

Implementing your own windowing would be a non-trivial amount of work, and spending the resources on computing both sets of windows sounds expensive.

Upvotes: 2

Related Questions