How copy resources files with sbt docker plugin

Question

I'm running an application using docker which crashes because my application cannot access to a file located in the folder src/main/resources. Here my file project/plugins.sbt :

addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.1.4")

my build.sbt file :

organization      := """foo"""

name              := """bar"""

version           := "1.0"

scalaVersion := "2.11.7"

libraryDependencies ++= Seq(
  "org.apache.kafka" % "kafka_2.10" % "0.10.0.0",
  "org.scalatest" % "scalatest_2.11" % "3.0.0-M16-SNAP1",
  "org.scalaz" %% "scalaz-core" % "7.2.5",
  "com.typesafe.akka" %% "akka-actor" % "2.4.11",
  "com.typesafe.play" % "play-json_2.11" % "2.4.8",
  "com.typesafe.scala-logging" %% "scala-logging" % "3.1.0",
  "com.typesafe" % "config" % "1.3.0",
  "com.amazonaws" % "aws-java-sdk" % "1.11.8",
  "ch.qos.logback" % "logback-classic" % "1.1.7",
  "com.typesafe.akka" % "akka-testkit_2.11" % "2.4.12"
)

// the default value in sbt is set to true 
parallelExecution in Test := false 

resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/conigtent/repositories/snapshots"

dependencyOverrides ++= Set(
  "com.fasterxml.jackson.core" % "jackson-databind" % "2.4.4"
)

resolvers ++= Seq(
  "Sonatype Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots/",
  "Sonatype Releases" at "https://oss.sonatype.org/content/repositories/releases/"
)

// http://www.scala-sbt.org/sbt-native-packager/formats/docker.html
enablePlugins(DockerPlugin, JavaAppPackaging)
// enablePlugins(JavaAppPackaging)
packageName in Docker := "foo/bar"

Here the main file :

object FGS_MainApp extends LazyLogging {


  val actorSystemName = "SYS"
  println("Create actor sys " + actorSystemName)
  val system = ActorSystem(actorSystemName)

  private val t = new Thread { 
    override def run() = {
      if (system != null) {
        Database.closeConnection
        system.terminate
        logger.info("ACTOR SYSTEM SHUTDOWN")
      }
    }        
  }

  Runtime.getRuntime.addShutdownHook(t)

  def main(args : Array[String]) = {
    // TODO : this needs to be remove quickly
    val saslPath = getClass.getResource("/sasl.conf").getPath
    println("saslPath = " + saslPath)
    System.setProperty("java.security.auth.login.config", saslPath)
  }
}

Then when I'm running the command : sbt docker:publishLocal && docker run -it foo/bar:1.0, I got this error :

Caused by: java.lang.SecurityException: java.io.IOException: Configuration Error:
    No such file or directory

The printed path is :

file:/opt/docker/lib/foo.bar-1.0.jar!/sasl.conf

What did I miss with the configuration in build.sbt ?

Update

Caused by: org.apache.kafka.common.KafkaException: java.lang.SecurityException: java.io.IOException: Configuration Error:
    No such file or directory
    at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86)
    at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:70)
    at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:623)
    ... 18 more
Caused by: java.lang.SecurityException: java.io.IOException: Configuration Error:
    No such file or directory
    at sun.security.provider.ConfigFile$Spi.<init>(ConfigFile.java:137)
    at sun.security.provider.ConfigFile.<init>(ConfigFile.java:102)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at javax.security.auth.login.Configuration$2.run(Configuration.java:255)
    at javax.security.auth.login.Configuration$2.run(Configuration.java:247)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.Configuration.getConfiguration(Configuration.java:246)
    at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:61)
    at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:46)
    at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:68)
    at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:78)
    ... 21 more
Caused by: java.io.IOException: Configuration Error:
    No such file or directory
    at sun.security.provider.ConfigFile$Spi.init(ConfigFile.java:335)
    at sun.security.provider.ConfigFile$Spi.init(ConfigFile.java:271)
    at sun.security.provider.ConfigFile$Spi.<init>(ConfigFile.java:135)
    ... 35 more

And it shows the line :

private val kafkaConsumer = new KafkaConsumer[String, Array[Byte]](kafkaProps) where kafkaProps is type of java.util.Properties. So it cannot find the file in resources folder.

Answer 1

The problem you're encountering is with how src/main/resources is treated. It's a special folder whose contents will be included in the jar file built by sbt. It's never actually copied as a file into your universal package. That's why the file path that's printed out includes the jar name and a ! - that means the file is inside the zipped jar. It's clear that the kafka library doesn't handle this.

To solve this, you should put the file in a different location, like src/main/conf (optional, but strictly-speaking more correct), then map it into your universal package:

mappings.in(Universal) +=
  ((sourceDirectory.value / "main" / "conf" / "sasl.conf"), "conf/sasl.conf"))

This will now be available as the file conf/sasl.conf within your universal package.