SBT: how to modify it:test with a task that calls the original test task?

Question

In the snippet of my Build.scala file below, the itTestWithService task starts a test server before and after running the integration tests.

I'd like to attach this itTestWithService task to the it:test key. But how?

  lazy val mohs =
    Project(id = "mohs", base = file("."))
     .settings (
        // I'd like the following but it creates a cycle that fails at runtime:
        // test in IntegrationTest <<= testWithService   
        itTestWithService <<= testWithService
      )

  val itTestWithService = taskKey[Unit]("run integration test with background server")

  /** run integration tests against a test server.  (the server is started before the tests and stopped after the tests) */
  lazy val testWithService = Def.task {
    val log = streams.value.log
    val launched = (start in testService).value
    launched match {
      case Success(_) =>
        testAndStop.value
      case Failure(e) =>
        val stack = e.getStackTrace().mkString("\n")
        log.error(s"failed to start test server: $e \n ${stack}")
    }
  }

  /** run integration tests and then stop the test server */
  lazy val testAndStop = Def.taskDyn {
    val _ = (test in IntegrationTest).value
    stop in testService
  }

Answer 1

In a related github issue discussion, Josh suggested an approach to the particular case I asked about (overriding it:test with a task that calls the original test task).

The approach works by reimplementing the test task. I don't know if there's a more general way to get access to the original version of the task. (A more general way would be a better answer than this one!)

Here's how to reimplement the it:test task:

  /** run integration tests (just like it:test does, but explicitly so we can overwrite the it:test key */
  lazy val itTestTask: Initialize[Task[Unit]] = Def.taskDyn {
    for {
      results <- (executeTests in IntegrationTest)
    } yield { Tests.showResults(streams.value.log, results, "test missing?") }
  }

Here's the composite integration test task (evolved slighly from the original question, though that version should work too):

  /** run integration tests against a test server.  (the server is started before the tests and stopped after the tests) */
  lazy val testWithServiceTask = Def.taskDyn {
    (start in testService).value match {
      case Success(_) =>
        testAndStop
      case Failure(e) =>
        val stack = e.getStackTrace().mkString("\n")
        streams.value.log.error(s"failed to start test server: $e \n ${stack}")
        emptyTask
    }
  }

  /** run integration tests and then stop the test server */
  lazy val testAndStop = Def.taskDyn {
    SbtUtil.sequence(itTestTask, stop in testService)
  }

  val emptyTask = Def.task {}

And now plugging the composite task we've built into the it:test key doesn't create a cycle:

  lazy val mohs =
    Project(id = "mohs", base = file("."))
      .settings (
        test in IntegrationTest <<= testWithServiceTask,
      )

Answer 2

You can add custom test tags in build.scala. Here's the sample code from one of my projects. Keep in mind, you don't have to bind it to it:test. You could name it anything you want.

lazy val AcceptanceTest = config("acc") extend(Test)

  lazy val Kernel = Project(
    id = "kernel",
    base = file("."),
    settings = defaultSettings ++ AkkaKernelPlugin.distSettings ++ Seq(
        libraryDependencies ++= Dependencies.Kernel,
        distJvmOptions in Dist := "-Xms256M -Xmx2048M",
        outputDirectory in Dist := file("target/dist"),
        distMainClass in Dist := "akka.kernel.Main system.SystemKernel"
      )
  ).configs(AcceptanceTest)
   .settings(inConfig(AcceptanceTest)(Defaults.testTasks): _*)
   .settings(testOptions in AcceptanceTest := Seq(Tests.Argument("-n",
     "AcceptanceTest"), Tests.Argument("-oD")))

Just pay attention to the lazy val at the top and the .configs part.

With that setup, when I type acc:test, it runs all tests with the AcceptanceTestTag. And you can just boot the server as part of what gets called by your test suite. Can even tag tests by needing server and not needing it so you can separate them once your suite gets big and takes longer to run.

Edit: Added to respond to comments.

To Tag tests, create a tag like this

import org.scalatest.Tag
object AcceptanceTest extends Tag("AcceptanceTest")

then put it here in your tests...

it("should allow any actor to subscribe to any channel", AcceptanceTest) {

And that cooresponds to the same build setup above. Only tests with That tag will be run when I call acc:test.

What came to mind for your problem was the solution I use for that same situation. Right now, you're doing the work in build.scala. I'm not sure if it's possible to do what you're saying right there... but what I do does the same thing but a bit differently. I have a trait that I mix-in to all tests that need a vagrant server. And I tag the tests that use it as VagrantTest.

And it works like a singleton. If one or more tests need it, it boots. But it will only boot one and all tests use it.

You could try doing the same, but over-ride it:test in the config file. Instead of "acc" in the example above, put "it". If that's not quite what you're looking for, may have to see if anyone else pops in.

So, basically when I call it:test it accesses running all tests with IntegrationTest test tag on it (as well as those with VagrantTest and a few others). So all the longer-running server tests don't get run as much (takes too long).