CODEWITHSUNDEEP
parametersapache-sparkapache-zeppelin
Andrei Stankevich
Andrei Stankevich

Reputation: 211

Run Spark with additional parameter through Zeppelin

I wand Zeppelin to start Spark with couple additional parameters (--total-executor-cores 200 --executor-memory 20g --driver-memory 5g) I am trying to add these parameters to spark interpreter through Zeppelin UI. I click edit button and add these three parameters but when I click save they disappear and I can see that my Spark was started by Zeppelin without these parameters.

Can you give me a tip how can I ask Zeppelin to start Spark with additional parameters?

Upvotes: 2

Views: 2477

Answers (1)

Greg
Greg

Reputation: 589

Take a look at conf/zeppelin-env.sh (or conf/zeppelin-env.sh.template). In the comments in that file under "Spark interpreter configurations" section:

# export SPARK_SUBMIT_OPTIONS                   # (optional) extra options to pass to spark submit. eg) "--driver-memory 512M --executor-memory 1G".

Upvotes: 3

Related Questions