CODEWITHSUNDEEP
elasticsearchapache-sparkcassandra
user2552806
user2552806

Reputation:

How to import elasticsearch-hadoop dependency to spark script

Im trying to use spark-es connector by setting the Build.scala document to:

libraryDependencies ++= Seq(
    "com.datastax.spark" %% "spark-cassandra-connector" % "1.2.1",
    "org.elasticsearch" %% "elasticsearch-hadoop" % "2.2.0"
  )

but I getting the error:

[error] (*:update) sbt.ResolveException: unresolved dependency: org.elasticsearch#elasticsearch-hadoop_2.10;2.2.0: not found

And I can see that it is exists here...

EDIT:

When I change my Build.scala to:

"org.elasticsearch" % "elasticsearch-hadoop" % "2.2.0"

I got the following error:

[error] impossible to get artifacts when data has not been loaded. IvyNode = org.scala-lang#scala-library;2.10.3
java.lang.IllegalStateException: impossible to get artifacts when data has not been loaded. IvyNode = org.scala-lang#scala-library;2.10.3

What is wrong?

Upvotes: 2

Views: 197

Answers (1)

user6022341
user6022341

Reputation:

elasticsearch-hadoop is not a Scala dependency so it doesn't have Scala specific version and cannot be used with %%. Try 

"org.elasticsearch" % "elasticsearch-hadoop" % "2.2.0"

Upvotes: 3

Related Questions