CODEWITHSUNDEEP
pythonmultiprocessingmxnet
Mkg Rty
Mkg Rty

Reputation: 39

Multiprocessing in mxnet gpu support

I have a model trained by mxnet gpu support and i load this model on gpu. My problem is that when I multiprocess I get an invalid pointer error.how to do multiprocessing in mxnet gpu support?

Upvotes: 0

Views: 415

Answers (1)

Olivier Cruchant
Olivier Cruchant

Reputation: 4037

MXNet is written in C++ and natively uses parallelism when relevant. You should rarely need to explicitly multiprocess it manually. You can use parallelism for CPU-led data transport operations (num_workers in gluon DataLoaders), or when sending inference payloads to an array of contexts such as here, but when facing a single context (like your single GPU), I don't think that things will behave well as it seems that CUDA struggles with multiprocessing (see here or here)

What I recommend instead is to leverage native MXNet and CUDA parallelism as much as possible, via:

  • the CPU: using a multiprocessed DataLoader (via num_workers) in training mode
  • the GPU: sending batches to the GPU. In training mode that's pretty normal, in inference mode that means doing asynchrony to enqueue requests and send batches to the GPU. Modern DL servers such as MMS allows this.

Upvotes: 2

Related Questions