CODEWITHSUNDEEP
javascalasbtapache-spark
chenzhongpu
chenzhongpu

Reputation: 6871

SBT: add third party library dependency but NoClassDefFoundError

I use sbt to build my Scala project.

Here is my build.sbt file:

name := "SpatialSpark"

version := "1.0"

scalaVersion := "2.10.4"

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.2.1"

libraryDependencies += "com.vividsolutions" % "jts" % "1.13"

Then I try to package it into jar. sbt package

Finally, I submit this to Apache-Spark,

spark-submit --class "com.chen.spatial.SpatialApp" --master local[4] target/scala-2.10/spatialspark_2.10-1.0.jar

It causes NoClassDefFoundError error,

Exception in thread "main" java.lang.NoClassDefFoundError: com/vividsolutions/jts/index/strtree/STRtree

Did I miss something when packing it into jar?

Upvotes: 1

Views: 617

Answers (2)

Michael Spector
Michael Spector

Reputation: 37019

Please look at this template project:

https://github.com/spektom/spark-scala-template

To build the uberjar run:

sbt assembly

To run the Jar in Spark:

$SPARK_HOME/bin/spark-submit --class com.github.spark.App spark-scala-assembly-0.0.1.jar

Upvotes: 2

Justin Pihony
Justin Pihony

Reputation: 67135

You should create an uber jar and after running

sbt assembly

it should work

Upvotes: 2

Related Questions