CODEWITHSUNDEEP
apache-sparkapache-spark-sqlcloudera
Dhruv Kapur
Dhruv Kapur

Reputation: 736

No SQL tab in spark history server for spark sql job

I am using Cloudera 5.8.3 distribution which is built with spark 1.6.0. I submit a Spark job through oozie (it is submitted as a java action)

I can see the job successfully complete and can also find it on the Spark history server(which is also running for Spark 1.6.0). I can also see all the executor/application/jobs/stages/tasks level detail.

However, I also know that spark provides better logging for spark dataframes, I don't see that tab. This is the view I'm talking about: Sql Tab

I'm unsure of what other details are relevant to this or where else to look for. Hoping to get some pointers.

Thanks!

Upvotes: 1

Views: 626

Answers (1)

zsxwing
zsxwing

Reputation: 20836

SQL Tab on history server is added in Spark 2.0.0. See https://issues.apache.org/jira/browse/SPARK-11206

Upvotes: 1

Related Questions