CODEWITHSUNDEEP
scalasbt
Max A.
Max A.

Reputation: 4892

How to vendor dependencies in an SBT project?

After ~8 years of using Scala, I took a detour to a small language you might have heard of called Go. It's not without its flaws (grow a real type system, boi!), but it does some things much better than Scala could ever hope to.

Go manages dependencies in source form, which any sensible engineer would consider terrifying until she discovers that storing one's dependencies in a vendor/ directory under source control is a "get out of jail free" card for cases when dependency resolution either becomes too complicated for its own good, or depends on flaky 3rd party resources, such as the network.

The latest version of Go's CLI tooling comes with a command called go mod vendor, which does the legwork of downloading the current module's dependencies into a vendor/ directory inside the project, which can subsequently be checked into source control. Setting aside discussions regarding the merits of aggressively and preemptively caching dependencies in this fashion, I would like to state for the record that this command is very convenient.

SBT is notorious for downloading dependencies into ~/.ivy2, which is more of a free-for-all cache shared by all of a user's projects rather than just one. There's a smaller cache in ~/.sbt, which is used by SBT itself as a Humpty Dumpty / Mr Potato Head scratch space. Both directories will be created & populated automatically if they don't exist, but neither is intended to be explicitly managed by the user. Both are internal implementation details of SBT and/or Ivy, and should not be messed with "unless you know what you're doing".

What I want (and now I'll be asking for things) is a sbt vendor command that would do the legwork of populating the unmanaged classpath with all of my project's dependencies. If it can also download all that's needed to run SBT itself into the same directory, that would be just peachy.

Is there a SBT plugin or some sequence of arcane incantations that can be used to accomplish that which I seek?

Upvotes: 0

Views: 385

Answers (1)

Max A.
Max A.

Reputation: 4892

Before this question gets closed forever by an overzealous posse of moderators, I'm going to post here the hack which got me over this particular hump. As usual, the solution ended up being a shell script:

#!/bin/bash

root="$(readlink -f "$(dirname "$0")")"
sbt="$root/.sbt-launcher"

if [ ! -x "$sbt" ]; then
  echo "$sbt does not exist or is not executable"
  exit 1
fi

exec "$sbt" \
  -ivy            "$root/.ivy2"          \
  -sbt-dir        "$root/.sbt"           \
  -sbt-boot       "$root/.sbt/boot"      \
  -sbt-launch-dir "$root/.sbt/launchers" \
  $@

Let's unpack this really quickly. First, the shell script is a wrapper for the real SBT launcher, which is located in the same directory and named .sbt-launcher. Any recent version should work; you too can download one from http://git.io/sbt.

My wrapper ensures that four flags are always passed to the real SBT launcher:

  • -ivy specifies a custom location for the Ivy cache.
  • -sbt-dir, -sbt-boot, and -sbt-launch-dir together force SBT to stop using the account-wide ~/.sbt directory as a dumping ground for SBT JARs and other things.

I saved this shell script as sbt inside my project, placed .sbt-launcher from http://git.io/sbt right next to it, and began using the wrapper instead of the real SBT. I then checked into source control the directories .ivy2 and .sbt which were created inside my project.

That's all it took. It's not an elegant solution, but it does well to isolate my CI/CD process from volatility in Internet artifact repositories.

Upvotes: 1

Related Questions