CODEWITHSUNDEEP
apache-flinkflink-streaming
Dhinesh
Dhinesh

Reputation: 325

How to submit the flink job other than using their dashboard?

I have written flink job to use data set and the data stream api. I have started both programs from the same main() method But now when I am submitting the job to flink Dashboard . only the dataset api program is running and the dashboard is saying job get finished and the stream is not triggered and the flink dashboard also not showing any this regarding datastream execution. But when I run from eclipse both the dataset and the datastream api program are running. Is there any other way to submit the job to flink job manager to do both the job?.

Upvotes: 1

Views: 5189

Answers (2)

Eron Wright
Eron Wright

Reputation: 1060

The solution is to split the two programs into separate classes, e.g. MyStreamingProgram and MyBatchProgram, each with a main method. Then, submit each program separately to Flink, so as to create two independent jobs.

You mentioned that you observed some variation in the behavior, depending on how the job was submitted. Under the hood, Flink uses various strategies for executing the main method to obtain the program plan. Producing numerous plans in a single execution of main is not supported (AFAIK).

Upvotes: 0

David Anderson
David Anderson

Reputation: 43454

Besides using the web dashboard, you can also submit jobs to Flink using the command line interface and the REST api.

However, it sounds like perhaps you are trying to have a single Flink job that uses both the DataSet and DataStream APIs. This won't work.

Upvotes: 1

Related Questions