CODEWITHSUNDEEP
apache-sparkspark-streaming
Pawan
Pawan

Reputation: 17

What should be the input to setJars() method in Spark Streaming

val conf = new SparkConf(true)
.setAppName("Streaming Example")
.setMaster("spark://127.0.0.1:7077")
.set("spark.cassandra.connection.host","127.0.0.1")
.set("spark.cleaner.ttl","3600")
.setJars(Array("your-app.jar"))

Lets say I am creating a Spark Streaming Application

What should be the content of "your-app.jar" file ? Do I have to create it manually in my local file system and pass the path or Is that a Scala compiled file using sbt.

If thats a scala file please help to write the code

Since I am a beginer I am just trying to run some sample codes.

Upvotes: 1

Views: 315

Answers (1)

Soumyadip Ghosh
Soumyadip Ghosh

Reputation: 188

setJars method of the SparkConf class takes external JARs that need to be distributed on the cluster. Any external drivers like JDBC, etc.

You do not have to pass your own application JAR in this if that's what you are asking.

Upvotes: 1

Related Questions