Scala future strange error on completion

Question

I am just getting started with futures and stumbled upon an error which seems quite strange to me:

Using play-ws to perform a post request and map the result:

wsClient.url(url).withHeaders("Content-Type" -> "application/json")
      .post(payload)
      .map { wsResponse =>
        if (!(200 to 299).contains(wsResponse.status)) {
          sys.error(s"Received unexpected status, open-cpu error ${wsResponse.status} : ${wsResponse.body}")
        }
        println(s"OK, received ${wsResponse.body}")

        wsResponse.json.validate[Seq[MyClass]] match {
          case JsSuccess(result, _) => result.map(outlierRes => Map("key" -> outlierRes.attr, "key2" -> outlierRes.attr2, "key3" -> outlierRes.val3))
          case JsError(error) => throw new MyException(error.toString())
        }
      }

works just fine. The println of the body shows everything is there, and the validation succeeds.

The problem lies here: aggregatedData = Await.result(theFutureFromAbove, 20.minutes)

This statement crashes with the following when run via the interactive console:

MyException
        at $anonfun$1.apply(<console>:44)
        at $anonfun$1.apply(<console>:44)
        at scala.util.Success$$anonfun$map$1.apply(Try.scala:206)
        at scala.util.Try$.apply(Try.scala:161)
        at scala.util.Success.map(Try.scala:206)
        at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
        at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
        at scala.concurrent.impl.ExecutionContextImpl$$anon$3.exec(ExecutionContextImpl.scala:107)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Which would suggest a parsing exception.

However, when run via sbt run there is a different exception:

java.lang.NullPointerException

Which resembles this line: wsClient.url(baseUrl + url).withHeaders("Content-Type" -> "application/json")

edit

It seems to be triggered by: wsClient.close() as if I closed the wsClient before the future completed. However, in the documentation it states

If you create a WSClient manually then you must call client.close() to clean it up when you’ve finished with it.

so where should I close it? Initially, I thought it would be safe to close after the Await.result but that still is throwing the error.

edit 2

wsResponse.json.validate[Seq[MyClass]].fold(
          error => {
            println(error)
            Future.failed(new MyException("parsing failed" + error))
          },
          result => result.map(data => Map("period" -> data.period, "amount" -> data.totalAmount, "outlier" -> data.isOutlier))
        )

But this still does not compile as Future[nothing] does not match my return type of Future[Seq[Map[String, Any]]]

Answer 1

Based on https://github.com/studiodev/Mocky/blob/master/app/services/GithubRepository.scala I finally found something similar to

private def parseGistResponse(response: WSResponse): Future[GistResponse] = {
    if (response.status < 400) {
      logger.debug(s"<< (${response.status}) ${response.body}")
      response.json.validate[GistResponse].fold(
        error => {
          logger.error(s"Unable to parse GistResponse: $error")
          Future.failed(new RuntimeException("parse-json-failed"))
        },
        gistResponse => Future.successful(gistResponse))
    } else {
      logger.error("Unable to parse GitResponse, cannot contact WS\n" + debugResponse(response))
      Future.failed(new RuntimeException("ws-error"))
    }
  }