CODEWITHSUNDEEP
hadoopapache-sparkoozie
Muqtafi Akhmad
Muqtafi Akhmad

Reputation: 35

Oozie stack trace

I'm triying to submit Spark job to Oozie using Spark's SparkSubmit Java class as suggested in

launching a spark program using oozie workflow but Java action failed with error message

Main class [org.apache.oozie.action.hadoop.JavaMain], exit code [1]

Is there any way to access the execution error log / stack trace so that I can see what really happened? Any comment / answer is appreciated.

Upvotes: 0

Views: 2450

Answers (1)

Paul H.
Paul H.

Reputation: 1074

To get to a java action log, you could use oozie's web console to find the hadoop job Id of that action. And then use Hadoop's Yarn WebUI to look into that hadoop job's mapper log.

With command line interface, the above steps would be:

  1. Run oozie cmd to get hadoop job id:

    oozie job -info <your job id here>

    and look for "External Id" in the output, such as : job_200904281535_0254

  2. Run yarn cmd to get hadoop job log:

    yarn logs -applicationId <your app id here>

    you can get app id by replacing "job" with "application" in job id, such as application_200904281535_0254

Upvotes: 3

Related Questions