CODEWITHSUNDEEP
javascalamavenapache-sparkintellij-idea
It'sOK
It'sOK

Reputation: 65

Apache Spark Maven POM errors

I am trying out Apache Spark and have a little problem that I am stuck with. I have this JavaSparkHiveExample.java example from the Apache Spark Github together with a pom.xml that I have created as a Maven project in IntelliJ IDEA.

I am able to run other Spark examples (using another simpler POM) but this one is giving me the following errors:

enter image description here

I am new with Maven and would therefore appreciate some things I could try in order to solve this problem.

Upvotes: 0

Views: 416

Answers (1)

CGS
CGS

Reputation: 2872

Issue is with the value of $project.version. It is referring to your project' version (2.3.0-snapshot). There isn't any maven dependency with this version in Maven central repository, hence you are facing this issue. Instead of using the project version, you can add a new property like this and refer it for all dependency version

<properties>
 <spark.version>1.6.2</spark.version>
</properties>

and then use it in the dependency 

<dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-core_${scala.binary.version}</artifactId>
      <version>${spark.version}</version>
    </dependency>

Make sure the version you are using is available in maven repository https://mvnrepository.com/ enter image description here

Upvotes: 1

Related Questions