CODEWITHSUNDEEP
apache-sparkplayframeworksbt
Simon
Simon

Reputation: 6480

Play! and Spark incompatible Jackson versions

I have a problem running Spark 2.1 with Play! 2.5.9.

I get the following runtime exception: 

com.fasterxml.jackson.databind.JsonMappingException: Incompatible Jackson version: 2.7.6

In the previous version of Spark (1.6), I managed to suppress this error adding in my build.sbt file:

dependencyOverrides ++= Set("com.fasterxml.jackson.core" % "jackson-databind" % "2.4.4")

But it doesn't work with Spark 2. I tried to change the version used in dependencyOverrides by 2.7.2 and 2.8.5, but I have no idea of which one I should use and why.

Any clue would be very appreciated ;)

Upvotes: 18

Views: 4363

Answers (3)

smallo
smallo

Reputation: 1025

I had the same problem with Spark 2.4.4 and play-json 2.8.1, so looked for the Spark dependencies in maven repository: https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.12/2.4.4 ; where I can see the version of jackson-databind that I need. Finally I've added the following line to my build.sbt file:

dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-databind" % "2.6.7.1"

This means play-json is now using an older version of jackson-databind which I don't really like, but it's working for me.

Upvotes: 0

Sal Borrelli
Sal Borrelli

Reputation: 2526

I had exactly the same problem with Spark 2.4.0. The dependency override that worked for me is:

dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-databind" % "2.9.8"

I believe the same override works for 2.4.1 as well.

Upvotes: 0

Simon
Simon

Reputation: 6480

Using a version 2.6.x in dependencyOverrides works (I still don't know if there is an other way to find a version that works than testing one after an other).

Upvotes: 18

Related Questions