CODEWITHSUNDEEP
scalaapache-sparkspark-streamingapache-spark-standalone
hatem dagbouj
hatem dagbouj

Reputation: 139

How to add the "--deploy-mode cluster" option to my scala code

209/5000 Hello I want to add the option "--deploy-mode cluster" to my code scala: 

  val sparkConf = new SparkConfig ().setMaster ("spark: //192.168.60.80:7077")

Without using the shell (the command. \ Spark-submit)

i whant to usage the " spark.submit.deployMode " in scala

Upvotes: 6

Views: 13323

Answers (2)

FaigB
FaigB

Reputation: 2281

with SparkConfig:

//set up the spark configuration and create contexts
val sparkConf = new SparkConf().setAppName("SparkApp").setMaster("spark: //192.168.60.80:7077")

val sc = new SparkContext(sparkConf).set("spark.submit.deployMode", "cluster")

with SparkSession:

val spark = SparkSession
   .builder()
   .appName("SparkApp")
   .master("spark: //192.168.60.80:7077")
   .config("spark.submit.deployMode","cluster")
   .enableHiveSupport()
   .getOrCreate()

Upvotes: 11

anshul_cached
anshul_cached

Reputation: 762

You can use 

 val sparkConf = new SparkConf ().setMaster ("spark: //192.168.60.80:7077").set("spark.submit.deployMode","cluster")

Upvotes: 4

Related Questions