CODEWITHSUNDEEP
apache-spark
Alex Luya
Alex Luya

Reputation: 9912

How to disable SparkUI programmatically?

I noticed that this bug has been fixed:

Allow users to disable Jetty Spark UI in local mode

https://issues.apache.org/jira/browse/SPARK-2100

but how to disable it programmatically?

is it by

sparkContext.setLocalProperty(?, ?)

I checked the documentation, but couldn't figure out which property to set

http://spark.apache.org/docs/latest/configuration.html#spark-ui

Upvotes: 23

Views: 15606

Answers (2)

Gokul
Gokul

Reputation: 493

@jacek - Even after enabling spark.ui.enabled to false in spark defaults ( verified it is available in node ) and still UI is accesible. Any clues? I am looking to disable the UI

Upvotes: 0

Josh Rosen
Josh Rosen

Reputation: 13801

Set spark.ui.enabled to false. This configuration isn't documented on Spark's configuration page because it was added as an internal configuration for debugging and unit tests [1].

With spark-submit, you could simply --conf spark.ui.enabled=false.

[1] https://github.com/apache/spark/pull/2363/files#r17436782

Upvotes: 46

Related Questions