CODEWITHSUNDEEP
scalaapache-sparksbt
Lubor
Lubor

Reputation: 999

After following the quick start of Spark, why do I get a SNAPSHOT jar file with no class file inside?

I'm following quick start of Spark 1.6 document, http://spark.apache.org/docs/latest/quick-start.html

at this step:

# Package a jar containing your application

$ sbt package

...

[info] Packaging {..}/{..}/target/scala-2.10/simple-project_2.10-1.0.jar

What I got is "simple_2.10-0.1-SNAPSHOT.jar" and there is no class inside this jar file. This will cause ClassNotFoundException in the next step when I run spark-submit command.

I have no idea what can cause this issue (why I got a jar file with "SNAPSHOT" and why there is no class in the jar file). Wondering if anyone can point this out. Thanks!

Upvotes: 0

Views: 257

Answers (1)

Justin Pihony
Justin Pihony

Reputation: 67115

First, the SNAPSHOT occurs because you did not specify a version in your sbt file.

Second, how do you know that there is no class inside your jar file? Did you run a jar tf on it? If you did and it is indeed empty, then my guess is that you did not store the source files in the expected directory structure, so it didn't pick it up. You can verify this by running sbt clean compile. You should see files being compiled if they are in the correct structure.

Upvotes: 1

Related Questions