CODEWITHSUNDEEP
multithreadingpytorchlibtorch
willem
willem

Reputation: 2717

How to disable multithreading in PyTorch?

I am trying to ensure that a PyTorch program build in c++ uses only a single thread. The program runs on CPU.

It has a fairly small model, and multi-threading doesn't help and actually causes problems because my program is multithreaded allready. I have called:

    at::set_num_interop_threads(1);
    at::set_num_threads(1);
    torch::set_num_threads(1);
    omp_set_num_threads(1);
    omp_set_dynamic(0);
    omp_set_nested(0);

In addition, I have set the environment variable OPENBLAS_NUM_THREADS to 1.

Still, when I spawn in single thread, a total of 16 threads show up on htop, and 16 of the processors of the machine go to 100%.

Am I missing something? What?

Upvotes: 7

Views: 3339

Answers (2)

gg99
gg99

Reputation: 656

Where you set the number of threads from matters: it should be in the same process that calls e.g. inference, not in the parent process.

It's hard to tell from the snippet you shared but that could be your issue.

There is a useful conversation there which should translate transparently to c++: https://discuss.pytorch.org/t/multiple-networks-running-in-parallel-on-different-cpus/70482/2

Upvotes: 0

mxbi
mxbi

Reputation: 893

From the PyTorch docs, one can do:

torch.set_num_threads(1)

To be on the safe side, do this before you instantiate any models etc (so immediately after the import). This worked for me.

More info: https://jdhao.github.io/2020/07/06/pytorch_set_num_threads/

Upvotes: 4

Related Questions