CODEWITHSUNDEEP
scalasbtsbt-assembly
EdgeCaseBerg
EdgeCaseBerg

Reputation: 2841

How to shade a dependency for a non-executable Scala Library?

Spent a few hours trying to figure out how to do this. Over the course of it I have looked at a few seemingly promising questions but none of them seem to quite fit what I'm doing.

I've got three library jars, let's call them M, S, and H. Library M has things like:

case class MyModel(x: Int, s: String)

and then library S uses the play-json library, version 2.3.8, to provide implicit serializers for the classes defined by M

trait MyModelSerializer {
    implicit val myModelFormt = Json.format[MyModel]
}

Which are then bundled up together into a convenience object for importing 

package object implicits extends MyModelSerializer extends FooSerizlier // etc

That way, in Library H, when it performs HTTP calls to various services it just imports implicits from S and then I call Json.validate[MyModel] to get back the models I need from my web services. This is all well and dandy, but I'm working on an application that's running play 2.4 and when I included H into the project and tried to use it I ran up against:

java.lang.NoSuchMethodError: play.api.data.validation.ValidationError.<init>(Ljava/lang/String;Lscala/collection/Seq;)

Which I believe is being caused by play 2.4 using play-json version 2.4.6. Unfortunately, these are a minor version apart and this means that trying to just use the old library like:

// In build.sbt
 "com.typesafe.play" %% "play-json" % "2.3.8" force()

Results in all the code in the app to fail to compile because I'm using things like JsError.toJson which weren't parts of play-json 2.3.8. I could change the 14 or so places trying to use that method, but given the exception before I have a feeling that even if I do that it's not going to help.

Around this point I remembered that back in my maven days I could shade dependencies during my build process. So I got to thinking that if I could shade the play-json 2.3.8 dependency in H that that would solve the problem. Since the problem seems to be that calling Json.* in H is using the Json object from play-json 2.4.6.

Unfortunately, the only thing I can find online that indicates the ability to shade is sbt-assembly. I found a great answer on how to do that for a fat jar. But I don't think I can use sbt-assembly because H isn't executable, it's just a library jar. I read through a question like my own but the answer refers to sbt-assembly so it doesn't help me.

Another question seems somewhat promising but I really can't follow how I would use it / where I would be placing the code itself. I also looked through the sbt manual, but nothing stuck out to me as being what I need.

I can't just change S to use play-json 2.4.6 because we're using H in a play 2.3 application as well. So it needs to be able to be used in both.

Right now the only thing I can really think to do if I can't get some kind of shading done is to make H not use S and to instead require some kind of serializer/deserializer implicitly and then wire in the appropriate json (dee)serializer. So here I am asking about how to properly shade with sbt with something that isn't an executable jar because I only want to do a re-write if I absolutely have to. If I missed something (like sbt-assembly being able to shade for non-executable jars as well), I'll take that as an answer if you can point me to the docs I must have missed.

Upvotes: 3

Views: 1990

Answers (1)

EdgeCaseBerg
EdgeCaseBerg

Reputation: 2841

As indicated by Yuval Itzchakov, sbt-assembly doesn't have to be building an executable jar and can shade library code as well. In addition, packing without transitive dependencies except the ones that need to be shaded can be done too and this will keep the packaged jar's size down and let the rest of the dependencies come through as usual.

Hunting down the transitive dependencies manually is what I ended up having to do, but if anyone has a way to do that automatically, that'd be a great addition to this answer. Anyway, this is what I needed to do to the H library's build file to get it properly shading the play-json library.

  1. Figure out what the dependencies are using show compile:dependencyClasspath at the sbt console
  2. Grab anything play related (since I'm only using play-json and no others I can assume play = needs shading)
  3. Also shade the S models because they rely on play-json as well, so to avoid transitive dependencies bringing a non-shadded play 2.3.8 back in, I have to shade my serializers.
  4. Add sbt-assembly to project and then update build.sbt file

build.sbt

//Shade rules for all things play:
assemblyShadeRules in assembly := Seq(
  ShadeRule.rename("play.api.**" -> "shade.play.api.@1").inAll
)

//Grabbed from the "publishing" section of the sbt-assembly readme, excluding the "assembly" classifier
addArtifact(artifact in (Compile, assembly), assembly)

// Only the play stuff and the "S" serializers need to be shaded since they use/introduce play:
assemblyExcludedJars in assembly := { 
  val cp = (fullClasspath in assembly).value
  val toIncludeInPackage = Seq(
    "play-json_2.11-2.3.8.jar", 
    "play-datacommons_2.11-2.3.8.jar", 
    "play-iteratees_2.11-2.3.8.jar", 
    "play-functional_2.11-2.3.8.jar", 
    "S_2.11-0.0.0.jar"
  )
  cp filter {c => !toIncludeInPackage.contains(c.data.getName)}
}

And then I don't get any exceptions anymore from trying to run it. I hope this helps other people with similar issues, and if anyone has a way to automatically grab dependencies and filter by them I'll happily update the answer with it.

Upvotes: 1

Related Questions