CODEWITHSUNDEEP
scalaakka
Philippe
Philippe

Reputation: 91

Akka messages all stop at once

I have this funny problem and I am unable to identify the issue.

I have this "simple" akka application. It's main goal is to go over every document in a database. My main actor requests rows to a single actor that communicates with the database. Each document retrieved is returned to my main actor. By batches, these documents are added to a message queue managed by a balancing dispatcher. Small workers will go over them and sort them.

After a few hours, usually between 2 and 4, all actors stop at the same time, up to 5 seconds interval.

I was wondering if any of you had every seen something similar.

For information:

Thank you for your help

From the DeadLetters it seems that only the actors related to my balancing dispatcher / my round robin router just stop. Would there be something I missed ?

My scala

val workers: ActorRef = context.system.actorOf(
  Props(new WorkerActor)
    .withRouter(FromConfig())
    .withDispatcher("balancing-dispatcher"),
  "round-robin"
)

My configuration code

balancing-dispatcher {
  type = BalancingDispatcher
  executor = "fork-join-executor"
}

akka.actor.deployment {
  /round-robin {
    router = round-robin
    nr-of-instances = 50
    resizer {
      lower-bound = 10
      upper-bound = 100
    }
  }
}

Upvotes: 1

Views: 294

Answers (1)

tangled-web
tangled-web

Reputation: 11

I would first use a profiling tool such as jconsole or jvisualvm to check for memory, GC, and/or fork+join issues. Do you have enough heap allocated? Also record the number of threads and thread states (are there threads being forked or joined when the slow-down occurs?)

It could be that you need to configure more threads in Akka's thread pool. Or that you've reached your upper limit of 100 instances and that they are all busy. You could subclass the DefaultResizer implementation to provide explicit notification/logging of resizer activity and configure your subclass as the resizer.

Upvotes: 1

Related Questions