CODEWITHSUNDEEP
hadoop-yarnapache-flink
Sohi
Sohi

Reputation: 424

Where to check logs for flink streaming job?

I have a flink streaming job continuously running on YARN. I am not able to see any application logs for running job . I only get logs when running as standalone java program.

Upvotes: 2

Views: 5627

Answers (2)

Sohi
Sohi

Reputation: 424

Well there is no direct way to check logs when running on YARN.

I have found an alternative . I have added Flink Rolling Sink to my streaming job which is writing logs to hdfs location with YYYY-mm-dd--HH format.

On top of that creating hive table to query on logs.

Upvotes: 1

ImbaBalboa
ImbaBalboa

Reputation: 867

The output of your Flink job will be written to the standard output on the TaskManagers.

EDIT : when the job is finished : On YARN, you might access that output by retrieving all logs using the aggregated YARN logs (yarn logs -applicationId <appid>) if you have enabled it.

A quick and dirty solution is to send your output in a sink socket for instance.

Upvotes: 0

Related Questions