CODEWITHSUNDEEP
pythonpython-2.7matplotlibram
Nils
Nils

Reputation: 930

RAM full when creating plots in loops

So I am just plotting a lot of plots, around 5000, and my RAM is running full. I've searched the site and web and found multiple solutions, such as plt.close, plt.clf() and gc.collect(), but neither help. I do not understand why my memory runs full even though I close all figures. Help would be much appreciated.

Same topic here but w/o a working solution for me: How can I release memory after creating matplotlib figures

y = range(5039*402) 
x = np.arange(0,402,1)
for i in xrange(len(data_collection)-1):
   plt.figure()
   plt.plot(x,y[i*402:402*(i+1)])
   plt.savefig('save%i.png'%(i))
   plt.close()

Sorry, might be an easy question, but I am new to python

Upvotes: 0

Views: 106

Answers (1)

DaveL17
DaveL17

Reputation: 2013

You don't mention which version of matplotlib you're using, but this is a known issue with earlier versions (it's supposed to be fixed in 2.0.x). One method I've used to "get around" the memory leak is to create each plot in its own process. For example, you can use a subprocess, thread or multiprocess for each plot.

Python subprocess

Python threading

Python multiprocessing

My preferred approach is multiprocessing because (IMO) it's far easier to move things in and out of each process (objects must be pickleable.)

Python pickling

ETA: Here is a silly example to show how you might structure your script using multiprocessing.

import datetime as dt
import multiprocessing as mp

return_queue1 = mp.Queue()
return_queue2 = mp.Queue()

def foo(text, return_queue):
    for _ in range(5000):
        continue
    return_queue.put(dt.datetime.now())

def bar(text, return_queue):
    for _ in range(5000):
        continue
    return_queue.put(dt.datetime.now())

for _ in range(5):
    log = 0
    if __name__ == '__main__':
        for _ in range(100):
            p1 = mp.Process(name='p1', target=foo, args=('foo', return_queue1,))
            p2 = mp.Process(name='p2', target=bar, args=('bar', return_queue2,))
            p1.start()
            p2.start()

            if return_queue1.get() > return_queue2.get():
                log += 1

        p1.join()
        p2.join()
        print(u"Times bar won: {0}".format(log))

Upvotes: 1

Related Questions