CODEWITHSUNDEEP
apache-sparkdriver
Suraj
Suraj

Reputation: 97

Spark: How to specify the IP for the driver program to run

I am having issue to configure specific spark node as driver in my cluster. I am having standalaone mode cluster. Every time on master restart i see that one of the node in the cluster is being randomly picked to run the driver program. Due to which i am enforced to deploy my JAR on all the nodes in my cluster.

If i can specify the IP for the driver program to run, then i need to deploy the JAR only in one node.

Appreciate, any help.

Upvotes: 1

Views: 940

Answers (1)

Vaibhavrtk
Vaibhavrtk

Reputation: 62

If you want to run from a particular node you can use: --deploy-mode client With this option the the driver program will always be running on the machine from where you run spark-submit.

For more information: http://spark.apache.org/docs/latest/submitting-applications.html#launching-applications-with-spark-submit

Upvotes: 2

Related Questions