CODEWITHSUNDEEP
scalasbt
llD
llD

Reputation: 11

Spark Scala SBT Task Failed

When I tray using spark with Scala in SBT build system to read a Json file a have the error:

and my SBT file is:

ThisBuild / version := "0.1.0-SNAPSHOT"

ThisBuild / scalaVersion := "2.13.10"


// https://mvnrepository.com/artifact/org.apache.spark/spark-core
libraryDependencies += "org.apache.spark" %% "spark-core" % "3.3.1"


// https://mvnrepository.com/artifact/org.apache.spark/spark-sql
libraryDependencies += "org.apache.spark" %% "spark-sql" % "3.3.1"

// https://mvnrepository.com/artifact/org.mongodb.spark/mongo-spark-connector
libraryDependencies += "org.mongodb.spark" % "mongo-spark-connector" % "10.0.5"

I tray to change all "3.3.1" and "10.0.5" to "3.0.1" and it still the same problem

Upvotes: 1

Views: 444

Answers (1)

ELinda
ELinda

Reputation: 2821

You should use a 2.12 version of Scala, since those dependencies do not all have 2.13 built.

ThisBuild / scalaVersion     := "2.12.11"

Look at the screenshot from Maven to see that 2.13 isn't listed for those versions (or at all).

Upvotes: 1

Related Questions