CODEWITHSUNDEEP
javascalaakkadeeplearning4j
Tom Piaggio
Tom Piaggio

Reputation: 669

Deeplearning4j Sharing Computational Graph between Threads in Scala

I'm trying to do image classification. I'm using Scala, the Akka actor system, and deeplearning4j. The thing is that I have to detect always on the same spots or crop on the image. I was thinking of creating a new actor for each crop of the image, on each frame. The thing is that, from what I understand, instantiating a new model for each actor creation is not viable, but having an instance of the model, and passing to each actor isn't either. Should I have a pool of instances? I'm a bit stuck with this problem, since it is the first time I'm trying deeplearning4j. Previously, I would use a python REST api, but I think that this solution should perform better.

Thank you in advance.

Upvotes: 1

Views: 228

Answers (3)

Adam Gibson
Adam Gibson

Reputation: 3205

In the future, just of note: You can use parallelinference for multi threaded serving out of the box. https://github.com/deeplearning4j/dl4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/inference/ParallelInferenceExample.java

One of the problems with the above answers, is it does not factor in that models have internal state when doing activations. ParallelInference also allows for both batched as well as sequential inference. This matters when you are using gpus for example. We allow multi threaded inference here as well.

Upvotes: 2

remi
remi

Reputation: 566

perhaps you could take a look at the monix library, it has async tasks, batching, fault-tolerance primitives, etc.

Upvotes: 0

Andrey Tyukin
Andrey Tyukin

Reputation: 44918

  1. There is no need to instantiate a new actor for each crop of the image. Simply keep a pool of actors that ask a master node to give them more images to classify as soon as they are done with the previous image. I also suggest to check whether it actually buys you anything compared to the ordinary parallel collections (sth. like images.par.map(model.classify) could already do the job; it would take care of a thread pool all by itself).
  2. There is no need to instantiate a new model for each actor. The classification does not mutate the model, so you can simply share the same model between all actors. That's like 8 bytes overhead per actor for a reference to the model object, therefore negligible.

Upvotes: 1

Related Questions