CODEWITHSUNDEEP
c++tensorflow
GPhilo
GPhilo

Reputation: 19173

How do I hide the GPU from one tf::Session() in a multi-session scenario?

I have a C++ program that makes use of two different classifiers running in separate tensorflow::Sessions. One of these models needs to run on the GPU, while the second is very small and I'd like for it to run on the CPU.

I tried creating the sessions using:

auto options = tensorflow::SessionOptions();
options.config.mutable_gpu_options()->set_per_process_gpu_memory_fraction(0);
m_session.reset(tensorflow::NewSession(options));

but this seems only to trigger the default "hog the GPU" behaviour.

I also tried playing around with options.config.mutable_gpu_options()->set_visible_device_list("-1") but there is a specific check in the TF code for invalid devices in the list that raises an error.

Setting CUDA_VISIBLE_DEVICES=-1 in the environment before running the program is of course not an option because the GPU should be visible for the session that needs it.

Does anyone know how to forbid just one one session to use the GPU?

Upvotes: 0

Views: 240

Answers (1)

M. Rath
M. Rath

Reputation: 71

An easy workaround would be to temporarily set CUDA_VISIBLE_DEVICES=-1 using putenv("CUDA_VISIBLE_DEVICES=-1"); and reset it after creating the session. 

std::string str_tmp = "CUDA_VISIBLE_DEVICES=";
str_tmp += getenv("CUDA_VISIBLE_DEVICES");
putenv("CUDA_VISIBLE_DEVICES=-1");

#Create session

#Reset environment variable
putenv(str_tmp);

However, there might be a cleaner way to do it without changing your environment variables.

Upvotes: 1

Related Questions