CODEWITHSUNDEEP
apache-sparkspark-streaming
Stefan Papp
Stefan Papp

Reputation: 2255

Error when creating a StreamingContext

I open the spark shell

spark-shell --packages org.apache.spark:spark-streaming-kafka_2.10:1.6.0

Then I want to create a streaming context

import org.apache.spark._
import org.apache.spark.streaming._


val conf = new SparkConf().setMaster("local[2]").setAppName("NetworkWordCount").set("spark.driver.allowMultipleContexts", "true")
val ssc = new StreamingContext(conf, Seconds(1))

I run into a exception:

org.apache.spark.SparkException: Only one SparkContext may be running in this JVM (see SPARK-2243). To ignore this error, set spark.driver.allowMultipleContexts = true. The currently running SparkContext was created at:

Upvotes: 6

Views: 2544

Answers (1)

Stefan Papp
Stefan Papp

Reputation: 2255

When you open the spark-shell, there is already a streaming context created. It is called sc, meaning you do not need to create a configure object. Simply use the existing sc object.

val ssc = new StreamingContext(sc,Seconds(1))

instead of var we will use val

Upvotes: 12

Related Questions