CODEWITHSUNDEEP
scalasbtsbt-assembly
Nandan Rao
Nandan Rao

Reputation: 343

assemblyMergeStrategy causing scala.MatchError when compiling

I'm new to sbt/assembly. I'm trying to resolve some dependency problems, and it seems the only way to do it is through a custom merge strategy. However, whenever I try to add a merge strategy I get a seemingly random MatchError on compiling:

[error] (*:assembly) scala.MatchError: org/apache/spark/streaming/kafka/KafkaUtilsPythonHelper$$anonfun$13.class (of class java.lang.String)

I'm showing this match error for the kafka library, but if I take out that library altogether, I get a MatchError on another library. If I take out all the libraries, I get a MatchError on my own code. None of this happens if I take out the "assemblyMergeStrategy" block. I'm clearly missing something incredibly basic, but for the life of me I can't find it and I can't find anyone else that has this problem. I've tried the older mergeStrategy syntax, but as far as I can read from the docs and SO, this is the proper way to write it now. Please help?

Here is my project/assembly.sbt:

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3")

And my project.sbt file: 

name := "Clerk"

version := "1.0"

scalaVersion := "2.11.6"

libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % "1.6.1" % "provided",
  "org.apache.spark" %% "spark-sql" % "1.6.1" % "provided",
  "org.apache.spark" %% "spark-streaming" % "1.6.1" % "provided",
  "org.apache.kafka" %% "kafka" % "0.8.2.1",
  "ch.qos.logback" %  "logback-classic" % "1.1.7",
  "net.logstash.logback" % "logstash-logback-encoder" % "4.6",
  "com.typesafe.scala-logging" %% "scala-logging" % "3.1.0",
  "org.apache.spark" %% "spark-streaming-kafka" % "1.6.1",
  ("org.apache.spark" %% "spark-streaming-kafka" % "1.6.1").
    exclude("org.spark-project.spark", "unused")
)

assemblyMergeStrategy in assembly := {
  case PathList("org.slf4j", "impl", xs @ _*) => MergeStrategy.first
}

assemblyOption in assembly := (assemblyOption in assembly).value.copy(includeScala = false)

Upvotes: 2

Views: 1482

Answers (1)

Yuval Itzchakov
Yuval Itzchakov

Reputation: 149558

You're missing a default case for your merge strategy pattern match:

assemblyMergeStrategy in assembly := {
  case PathList("org.slf4j", "impl", xs @ _*) => MergeStrategy.first
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
   oldStrategy(x)
}

Upvotes: 4

Related Questions