CODEWITHSUNDEEP
apache-spark
Kannan
Kannan

Reputation: 307

spark-submit Error: No main class set in JAR; please specify one with --class

I am trying to execute a spark application using spark-submit.

c:\temp>spark-submit --master yarn  ./SparkExamples.jar --class com.examples.WordCount
Error: No main class set in JAR; please specify one with --class
Run with --help for usage help or --verbose for debug output

The main class file is existing in the jar file. I also checked there is a MANIFEST.MF file which holds the main class name.

Manifest-Version: 1.0

Main-Class: com.examples.WordCount

what am I missing?

Upvotes: 9

Views: 15883

Answers (2)

Shawn Doherty
Shawn Doherty

Reputation: 31

If you are including multiple jars as dependencies, make sure they are comma separated without spaces. Took a while to figure out, but this was my issue, and the error messages was the opposite of helpful since the class name was clearly already present.

Here is an example below of what fixed it.

Example Code causing error:

spark-submit --jars jar1.jar, jar2.jar \
--class ClassName \
jar3.jar param1 param 2

Example code fixed:

spark-submit --jars jar1.jar,jar2.jar \
--class ClassName \
jar3.jar param1 param 2

Upvotes: 3

hd16
hd16

Reputation: 262

Try providing --class parameter first before ./SparkExamples.jar

spark-submit --master yarn  --class com.examples.WordCount ./SparkExamples.jar

Upvotes: 13

Related Questions