CODEWITHSUNDEEP
apache-sparkapache-zeppelinapache-spark-1.5
Wanchun
Wanchun

Reputation: 165

How to connect Zeppelin to Spark 1.5 built from the sources?

I pulled the latest source from the Spark repository and built locally. It works great from an interactive shell like spark-shell or spark-sql.

Now I want to connect Zeppelin to my Spark 1.5, according to this install manual. I published the custom Spark build to the local maven repository and set the custom Spark version in the Zeppelin build command. The build process finished successfully but when I try to run basic things like sc inside notebook, it throws:

akka.ConfigurationException: Akka JAR version [2.3.11] does not match the provided config version [2.3.4]

Version 2.3.4 is set in pom.xml and spark/pom.xml, but simply changing them won’t even let me get a build.

If I rebuild Zeppelin with the standard -Dspark.vesion=1.4.1, everything works.

Upvotes: 11

Views: 1395

Answers (1)

bzz
bzz

Reputation: 663

Update 2016-01

Spark 1.6 support has landed to master and is available under -Pspark-1.6 profile.

Update 2015-09

Spark 1.5 support has landed to master and is available under -Pspark-1.5 profile.

Work on supporting Spark 1.5 in Apache Zeppelin (incubating) was done under this PR apache/incubator-zeppelin#269 which will lend to master soon.

For now, building from Spark_1.5 branch with -Pspark-1.5 should do the trick.

Upvotes: 8

Related Questions