CODEWITHSUNDEEP
apache-flink
Dagang Wei
Dagang Wei

Reputation: 26548

Flink example job of long-running streaming processing

I'm looking for a Flink example job of long-running streaming processing for test purposes. I checked the streaming/WordCount included in the Flink project, but seems it is not long-running, after processing the input file, it exits.

Do I need to write one by myself? What is the simplest way to get an endless stream input?

Upvotes: 2

Views: 1089

Answers (2)

David Anderson
David Anderson

Reputation: 43707

The WordCount example exits because its source is finite. Once it has fully processed its input, it exits.

The Flink Operations Playground is a nice example of a streaming job that runs forever.

Upvotes: 3

TobiSH
TobiSH

Reputation: 2921

Per definition every streaming job runs "forever" as long as you don't define any halt criterias or cancel the job manually. I guess you are asking for some job which consumes from some kind of infinite source. The most simplest job I could find was the twitter example which is included at the flink-project itself:

https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/scala/org/apache/flink/streaming/scala/examples/twitter/TwitterExample.scala

With some tweaking you could also use the socket-example (there you have some more control of the source):

https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/scala/org/apache/flink/streaming/scala/examples/socket/SocketWindowWordCount.scala

Hope I got your question right and this helps.

Upvotes: 1

Related Questions