CODEWITHSUNDEEP
rsparkr
Danny M.
Danny M.

Reputation: 281

sparkR Installation Issue 1.4.1

I have tried the following for both spark 1.4.0 and 1.4.1 on a Mac. I am downloading the package type = 'Source Code [can build several Hadoop versions' and download type http://ftp.wayne.edu/apache/spark/spark-1.4.1/spark-1.4.1.tgz.

When I run ./bin/sparkR in the terminal, I get an error message saying "~assembly/target/scala-2.10" not found. Any ideas?

Upvotes: 0

Views: 179

Answers (1)

Shivaram Venkataraman
Shivaram Venkataraman

Reputation: 249

If you want to directly use SparkR, you will need to download the binary package pre-built for a specific Hadoop version. To run on your laptop you can use any Hadoop version so say http://www.apache.org/dyn/closer.cgi/spark/spark-1.4.1/spark-1.4.1-bin-hadoop2.4.tgz (which is for Hadoop 2.4) should be fine.

If you want to use the source package, then you need to build Spark before running the script. You can do that with a command that looks like build/mvn -DskipTests -Psparkr package. More instructions are at https://github.com/apache/spark/tree/master/R#sparkr-development and http://spark.apache.org/docs/latest/building-spark.html

Upvotes: 1

Related Questions