CODEWITHSUNDEEP
scalaapache-sparkspark-streaming
Beyhan Gul
Beyhan Gul

Reputation: 1259

Spark Streaming StreamingContext error

Hi i am started spark streaming learning but i can't run an simple application My code is here 

    import org.apache.spark._
    import org.apache.spark.streaming._
    import org.apache.spark.streaming.StreamingContext._
    val conf = new SparkConf().setMaster("spark://beyhan:7077").setAppName("NetworkWordCount")
    val ssc = new StreamingContext(conf, Seconds(1))
    val lines = ssc.socketTextStream("localhost", 9999)
    val words = lines.flatMap(_.split(" "))

And i am getting error like as the following

scala> val newscc = new StreamingContext(conf, Seconds(1))
15/10/21 13:41:18 WARN SparkContext: Another SparkContext is being constructed (or threw an exception in its constructor).  This may indicate an error, since only one SparkContext may be running in this JVM (see SPARK-2243). The other SparkContext was created at:

Thanks

Upvotes: 2

Views: 7588

Answers (2)

mniehoff
mniehoff

Reputation: 517

It looks like you work in the Spark Shell. There is already a SparkContext defined for you there, so you don't need to create a new one. The SparkContext in the shell is available as sc

If you need a StreamingContext you can create one using the existing SparkContext:

val ssc = new StreamingContext(sc, Seconds(1))

You only need the SparkConf and SparkContext if you create an application.

Upvotes: 2

mehmetminanc
mehmetminanc

Reputation: 1379

If you are using spark-shell, and it seems like you do, you should not instantiate StreamingContext using SparkConf object, you should pass shell-provided sc directly.

This means:

val conf = new SparkConf().setMaster("spark://beyhan:7077").setAppName("NetworkWordCount")
val ssc = new StreamingContext(conf, Seconds(1))

becomes,

val ssc = new StreamingContext(sc, Seconds(1))

Upvotes: 5

Related Questions