CODEWITHSUNDEEP
c++tensorflowmemory-managementgpu
7oud
7oud

Reputation: 75

How to change per_process_gpu_memory_fraction in tensorflow using c++?

I have develop two applications based on tf in c++ language, these applications are served as libraries. In the caller execuable program, library1 is called then library2. In library1 initialization, gpu memory fraction is set to 0.5, run some inference, and session closed. then library2 is called, gpu memory fraction is set to 0.8, but the setting can not work, gpu memory allocation did not change. Both library have the same initialization code but differnet fraction value

int XXXLib::init(double per_process_gpu_memory_fraction)
{
    SessionOptions options;
    ConfigProto* config = &options.config;
    GPUOptions* gpu_options = config->mutable_gpu_options();

    // for library1, fraction = 0.5; for library2, fraction = 0.8
    gpu_options->set_per_process_gpu_memory_fraction(per_process_gpu_memory_fraction);
    Status status = NewSession(options, &_session);
}

It seems that when set_per_process_gpu_memory_fraction() is called, the gpu memory in this process is fixed, even new another Newsession(), the original fraction value is used.

  1. Should different app(library) use different session ?
  2. gpu memory fraction is related to session or to process ?
  3. How to change the fraction in different session but the same process?

Some env info:

Upvotes: 2

Views: 1821

Answers (1)

iga
iga

Reputation: 3633

It is unfortunate, but in the current TensorFlow (1.11) the GPU memory allocator is created once (per GPU device) - the first time a session is created in the process. Changing per_process_gpu_memory_fraction in the following sessions will not have any effect.

In regards to your library, I would suggest not creating sessions inside it. Ask the user to provide you a session that they configure as they wish. Alternatively, you can just create a graph and return the operations to run. The user can then run them as they see fit.

Upvotes: 1

Related Questions