CODEWITHSUNDEEP
scalamavenapache-sparkmaven-2
HHH
HHH

Reputation: 6455

Building Spark using Maven fails

I have a spark program written in Scala and I'm trying to build it using Maven. However maven built fails without any obvious error. Here is the error message:

  [INFO] Compiling 1 source files to D:\Scala-IDE\Workspace\ClassifierMaven\classifier\target\classes at 1438214677096
[ERROR] error: class file needed by package is missing.
[INFO] reference type ClassTag of package reflect refers to nonexisting symbol.
[ERROR] one error found
[INFO] Picked up _JAVA_OPTIONS: -Xmx2G
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17.022 s
[INFO] Finished at: 2015-07-29T17:04:41-07:00
[INFO] Final Memory: 30M/879M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.0:compile (default) on project classifier: wrap: org.apache.commons.exec.ExecuteException: Process
exited with an error: 1(Exit value: 1) -> [Help 1]

what is the problem?

SOLUTION: the problem was missing a dependency in my POM. I added the dependency and now I'm getting a new error message. HEre is the error I'm getting now

[INFO] excludes = []
[INFO] D:\Scala-IDE\Workspace\ClassifierMaven\classifier\src\main\scala:-1: info: compiling
[INFO] Compiling 1 source files to D:\Scala-IDE\Workspace\ClassifierMaven\classifier\target\classes at 1438274292378
[INFO] java.lang.reflect.InvocationTargetException
[INFO]  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[INFO]  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
[INFO]  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[INFO]  at java.lang.reflect.Method.invoke(Method.java:606)
[INFO]  at org_scala_tools_maven_executions.MainHelper.runMain(MainHelper.java:161)
[INFO]  at org_scala_tools_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26)
[ERROR] Caused by: java.lang.AssertionError: assertion failed: List(object Byte, object Byte)
[INFO]  at scala.tools.nsc.symtab.Symbols$Symbol.suchThat(Symbols.scala:1063) 
[INFO]  at scala.tools.nsc.symtab.Symbols$Symbol.companionModule0(Symbols.scala:1269)
[INFO]  at scala.tools.nsc.symtab.Symbols$Symbol.companionModule(Symbols.scala:1277)
[INFO]  at scala.tools.nsc.symtab.Symbols$Symbol.linkedClassOfClass(Symbols.scala:1296)
[INFO]  at scala.tools.nsc.symtab.Definitions$definitions$.addModuleMethod$1(Definitions.scala:707)
[INFO]  at scala.tools.nsc.symtab.Definitions$definitions$.initValueClasses(Definitions.scala:710)
[INFO]  at scala.tools.nsc.symtab.Definitions$definitions$.init(Definitions.scala:787)
[INFO]  at scala.tools.nsc.Global$Run.<init>(Global.scala:597)
[INFO]  at scala.tools.nsc.Main$.process(Main.scala:107)
[INFO]  at scala.tools.nsc.Main$.main(Main.scala:122)
[INFO]  at scala.tools.nsc.Main.main(Main.scala)
[INFO]  ... 6 more
[INFO] Picked up _JAVA_OPTIONS: -Xmx2G

Upvotes: 0

Views: 600

Answers (2)

Shemeem
Shemeem

Reputation: 198

I got same issue, check your java version in your pom. it must be 1.8

Upvotes: 0

Holden
Holden

Reputation: 7452

It seems that you are missing the scala-reflect package from your class path. I'd suggest looking at some of the sample pom's (like learning spark, or some of the other databricks ones) and using one of those as a starting point for your build.

Upvotes: 1

Related Questions