CODEWITHSUNDEEP
pythontensorflowipython
DDz
DDz

Reputation: 57

When run a tensorflow session in iPython, GPU memory usage remain high when exiting iPython

I think it's some sort of bug.

The problem is quite simple:

  1. launch ipython

  2. import Tensorflow and run whatever session

  3. type nvidia-smi in bash (see really high gpu memory usage, related process name, etc)

  4. control+z quit ipython

  5. type nvidia-smi in bash (still! really high GPU memory usage, and the same process name, strangely, these processes are not killed!)

I guess iPython failed to clean Tensorflow variables or graphs when exiting.

Is there any way I can clean the GPU memory without restart my machine?

System: Ubuntu 14.04
Python: Python3.5
IPython: IPython6.0.0

Upvotes: 0

Views: 499

Answers (1)

ales_t
ales_t

Reputation: 2017

Control+Z doesn't quit a process, it stops it (use fg to bring it back up). If some computation is running in a forked process, it may not stop with the main process (I'm no OS guy, this is just my intuition).

In any case, properly quitting iPython (e.g. by Control+D or by running exit()) should solve the problem. If you need to interrupt a running command, first hit Control+C, then run exit().

Upvotes: 3

Related Questions