CODEWITHSUNDEEP
scalaactor
prosseek
prosseek

Reputation: 190979

With Scala, what's the (practical) limit number of Actors in a single process?

I have a simulation code that creates N samples, and process it one by one. With IoC (Inversion of control), the processor method is given a method to be invoked. The number of samples are controlled from the configuration["iteration"] map.

This is the processor method:

def processor(configuration:Map[String, Any], f:(Int, Summary) => Unit) = {
    val byteWidth:Int = configuration.getOrElse("byteWidth", 4).asInstanceOf[Int]
    ...
    val iteration:Int = configuration.getOrElse("iteration", 10000).asInstanceOf[Int]

    val caller = self
    (1 to iteration).foreach { i =>
            ...
            val newBf = Summary.makeIt() // generate simulation data
            f(i, newBf)
        }
    }

}

This is the caller functions:

val conf = MMap[String, Any]()
conf("iteration") = 100000
def calculate(i:Int, bf: Summary) : Unit = {
  ... 
}
processor(configuration = conf.toMap, calculate)

This code works fine, but it does not use multi-core, so I modified the processor to use actor.

def parallelProcess(configuration:Map[String, Any], f:(Int, Summary) => Unit) = {
    ...
    val iteration:Int = configuration.getOrElse("iteration", 10000).asInstanceOf[Int]

    val caller = self
    (1 to iteration).foreach { i =>
      actor {
        caller ! {
            ...
            val newBf = Summary.makeIt() // generate simulation data
            f(i, newBf)
        }
      }
    }
    (1 to iteration).foreach { i =>
      receive {
        case msg => msg
      }
    }

}

This code works fine, and it uses all the cores that I have. Even when I create 100K samples (and accordingly actors), it works OK, but it slows down with 1 million actors, it slows down a bit, and 10 million samples it becomes very slow with all the cores are busy.

I expect that too may actors may be the culprit, even 100K actors seem pretty large number already, but works fine.

How many actors are too much actors? Is there way to control the number of actors for this kind of problem?

Upvotes: 0

Views: 672

Answers (2)

Lanny Ripple
Lanny Ripple

Reputation: 419

An actor is a lightweight unit of control. It's basically a PartialFunction describing how to respond to a message and a mailbox containing messages. A central event loop takes a message out of the mailbox, sees if the actor will respond to it, and if so schedules the execution to happen on a thread. Threads are a bit heavier. They weigh in at about 1MB of memory per thread. Among perfectly CPU-bound processing you really only need 1 thread per core.

The question of how many actors are too many will depend on your application. You'd basically need to figure out how much cpu time per message you use on average and how many cores you have. If say 20ms per message with 8 cores you could provide a threadpool of size 8 and 400 actors and you should be maxing out all your cpu time. Just because you can provide more actors and/or more threads doesn't mean you'll get things done quicker. You'll reach a minimum and then start adding time to your runs as the overhead of managing the threads and actors starts to come into play.

You can modify Akka's configuration to control the threadpool size or you manage custom execution contexts. See the very fine docs for details.

Upvotes: 4

Ravi Kiran
Ravi Kiran

Reputation: 1139

The Scala standard library actors have been deprecated and it is recommended to use Akka actors from Scala 2.10 onwards. Akka claims to support 2.5 million actors per GB of memory (~400 bytes of overhead per actor).

I don't think there is any inbuilt mechanism to put an upper limit on the number of actors in a process. You have to build application logic for that.

FWIW, I have run an Akka node with 240,000 actors with high throughput and no visible slowdown on a MackBook Pro.

Upvotes: 1

Related Questions