CODEWITHSUNDEEP
c++tensorflowmemorygpu
Mark Davey
Mark Davey

Reputation: 51

Using Tensorflow in C++ application: How to release GPU memory

I am trying to use Tensorflow for inference within my C++ application. Other parts of the application need access to large amounts of GPU memory (not at exactly the same time as Tensorflow). However, once Tensorflow has been used to perform inference, it hogs the GPU memory and does not release it until the application ends. Ideally, after inference, I would be able to free the GPU memory used by Tensorflow to allow other algorithms to use the GPU.

Has anyone else faced this problem, and did you find a solution?

Upvotes: 2

Views: 492

Answers (1)

Max Vollmer
Max Vollmer

Reputation: 8598

Tensorflow allocates memory for the lifetime of the process. There is unfortunately no way around that, you only get the memory back once the process finishes.

One way to solve this would be to "modularize" your application into multiple distinct processes. Have one process for performing inference, and a parent process (your application) which calls it. You can run the child process blocking, so your entire app behaves as if it was executing the code itself (apart from handling resource sharing of course).

Upvotes: 2

Related Questions