CODEWITHSUNDEEP
scalaplayframework-2.0
Michael Tiller
Michael Tiller

Reputation: 9421

Async Request in (Scala) Play "Hangs"

I've written an application (Scala) Play and when I run it on my local machine, it works exactly as I expect. The method that I'm working on (there may be others with this issue, I haven't checked extensively) is a method triggered by a POST request. It returns an Async object because the evaluation done by the method returns a Future.

But this method behaves quite differently when I deploy it. I've been chasing this down all day, but I've gotten to the point where I have no explanation for what is going on.

The method looks something like this:

  def add = Action(parse.json) { request =>
    Async {
      val req = request.body.asOpt[TaskRequest] map { r =>
        val job = createJob(r)
        submitJob(job, r)
        job map { ij =>
          allowOrigin(Created(ij).withHeaders("Location" -> routes.Jobs.read(ij._id.stringify).url));
        }
      }
      req.get
    }
  }

Overall, this is pretty simple. The createJob method returns a Future[Job]. The submitJob call is a kind of "fire and forget" call to trigger some background stuff (that I'm not interested in waiting for). I then process the job result to produce a Result object when the job has been created.

All this works as I described above on my local machine. But when I deploy this, something really odd happens. That submitJob call triggers a long running computation. As I said, I'm not interested in waiting for it which is why I don't even use the Future that it returns anywhere.

But here's the odd part...on my development machine, the response from my POST request (which directly calls add) comes back as soon as the job is created. But on the deployment machine, I only get a response after the submitJob task is completed! In other words, I've got this req as a Future that is seemingly completely unrelated to that submitJob call (it certainly doesn't depend on its outcome), but in the deployment environment, Async block won't return until submitJob is complete (even though req is fulfilled almost immediately).

What the heck is going on here? I've put println statements all through the code. It definitely gets past that req.get call almost immediately. But the overall (HTTP) response definitely waits until submitJob is done. Does Async somehow know about the submitJob call? IF so, why does it behave differently on two different machines?

Any ideas?

Upvotes: 3

Views: 1016

Answers (2)

Jatin
Jatin

Reputation: 31724

Though you have figured out, but there is an another alternative. This normally happens when you use say scala's default ThreadPool i.e. scala.concurrent.ExecutionContext.Implicits.global. It uses a newFixedSizeThreadPool with size as number of cores of machine.

An alternative would be to use your own ExecutionContext with the future calls above. i.e. in your above code, add this line:

implicit val exec = ExecutionContext.fromExecutor(Executors.newCachedThreadPool())

All your future calls will then use the above ExecutionContext.

Upvotes: 4

Michael Tiller
Michael Tiller

Reputation: 9421

OK, I figured it out. Oh what a pain that was. The issue was that my "fire and forget" task was a long running task. But the thread pool size on the deployment machine ended up being smaller than my development machine. As a result, this long running task dropped in the thread pool and clogged it up. So the processing of the AsyncResult was presumably preempted by the long running calculation (it was actually blocking on a result from a long running calculation, but the bottom line is that it blocked).

After reading a really nice explanation of futures in Scala 2.10 by Heather Miller, I noticed the blocking construct that can be used to temporarily expand the size of the thread pool for a single future. I wrapped my blocking code with a blocking call and that did the trick. Presumably, this allocated an additional temporary worker to handle my blocking call and, as a result, the AsyncResult stuff got processed in a timely manner.

(Note, wrapping the submitJob call was useless because that, itself, didn't block. What I had to do was go into the submitJob call and find the place where the actual blocking was and wrap that.)

Upvotes: 4

Related Questions