CODEWITHSUNDEEP
apache-spark
Sergei Russkikh
Sergei Russkikh

Reputation: 123

Run spark from source code on Windows - no such file or directory error

I would like to run Spark from source code on my Windows machine. I did the following steps:

  1. git clone https://github.com/apache/spark
  2. Added the SPARK_HOME variable into the user variables.
  3. Added %SPARK_HOME%\bin to the PATH variable.
  4. ./build/mvn -DskipTests clean package
  5. ./bin/spark-shell

The last command returns the following error: enter image description here

What should I do to fix the error?

Upvotes: 0

Views: 330

Answers (1)

sjyang
sjyang

Reputation: 36

First, refer to the link below for the solution. The top voted answer gave me the working script for this problem.
: Failed to start master for Spark in Windows

The reason is that spark launch scripts do not support Windows. The spark documentation (https://spark.apache.org/docs/1.2.0/spark-standalone.html) insists you to start the master and workers manually if you are a Windows user. So you need to first run the master and then run spark-shell.

Upvotes: 2

Related Questions