CODEWITHSUNDEEP
streamingspark-java
Ash
Ash

Reputation: 3550

StreamingContext couldn't bind to a port used by Java

I have started Spark master and workers and can easily run a MapReduce like wordcount on HDFS.

Now I want to run a streaming on textstream and when I want to make a new StreamingContext I have this error:

scala> val ssc = new StreamingContext("spark://master:7077","test", Seconds(2))
13/07/17 11:13:45 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started
org.jboss.netty.channel.ChannelException: Failed to bind to: /192.168.2.105:48594
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:298)
....

I checked the port and it was used by Java. I killed the process and I got out of Spark-shell.

Is there any way I can change the StreamingContext's port to a random free port?

Upvotes: 1

Views: 327

Answers (1)

WestCoastProjects
WestCoastProjects

Reputation: 63062

Java is the underlying process for spark (scala runs on the jvm). It is possible that you have multiple copies of spark /spark streaming running. Can you look into that?

Specifically: i get the same result if I have a spark-shell already running.

You can check for other spark processes:

ps -ef | grep spark | -v grep

Upvotes: 1

Related Questions