CODEWITHSUNDEEP
apache-sparkapache-spark-1.6
Naresh
Naresh

Reputation: 5397

Running spark job not shown in the UI

I have submitted my spark job as mentioned here bin/spark-submit --class DataSet BasicSparkJob-assembly-1.0.jar without mentioning the --master parameter or spark.master parameter. Instead of that job gets submitted to my 3 node spark cluster. But i was wondering where it submitted the job because it is not showing any information in the Running Applications

Upvotes: 6

Views: 8765

Answers (2)

user1314742
user1314742

Reputation: 2924

If you do not set the master in --master nor spark.master Spark will run locally.

You could still view the progress of your job. By default the UI will be availalbe during the running of your spark job on http://localhost:4040.

When your job finishes, this UI will be killed and you could not view the history of your application unless you configured Spark history server

Upvotes: 5

Kien Truong
Kien Truong

Reputation: 11381

It's likely that Spark is running your in local mode on your development machine.

Upvotes: 2

Related Questions