CODEWITHSUNDEEP
scalaapache-sparkamazon-emr
Alastair McCormack
Alastair McCormack

Reputation: 27744

Spark on AWS EMR: java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V

I'm trying to deploy a Scala 2.12.0 + Spark 2.4.4 application to AWS EMR. I have Spark 2.4.4 enabled in EMR.

When the application runs, it fails with the following error:

java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V

I'm building the jar using the "sbt-assembly" plugin. I specifically remove "org/apache/spark/" from the assembled jar to ensure no conflict in versions.

Upvotes: 4

Views: 813

Answers (1)

Alastair McCormack
Alastair McCormack

Reputation: 27744

While Spark 2.4.4 supposedly supports Scala 2.12.x, EMR doesn't seem to support it.

Instead, one must build using Scala 2.11.

E.g. in the build.sbt file:

scalaVersion := "2.11.12"

Upvotes: 5

Related Questions