CODEWITHSUNDEEP
apache-spark
kamalbanga
kamalbanga

Reputation: 2011

java.net.ConnectException (on port 9000) while submitting a spark job

On running this command:

~/spark/bin/spark-submit --class [class-name] --master [spark-master-url]:7077 [jar-path]

I am getting

java.lang.RuntimeException: java.net.ConnectException: Call to ec2-[ip].compute-1.amazonaws.com/[internal-ip]:9000 failed on connection exception: java.net.ConnectException: Connection refused

Using spark version 1.3.0.

How do I resolve it?

Upvotes: 0

Views: 1041

Answers (1)

kamalbanga
kamalbanga

Reputation: 2011

When Spark is run in Cluster mode, all input files will be expected to be from HDFS (otherwise how will workers read from master's local files). But in this case, Hadoop wasn't running, so it was giving this exception.

Starting HDFS resolved this.

Upvotes: 2

Related Questions