CODEWITHSUNDEEP
pythonmemory-leaks
ilyaw77
ilyaw77

Reputation: 1047

python default dict memory leak

I'm not sure if it is a memory leak but looks like one and I cannot find a solution here. I cannot paste the full code but here is the part that leaking:

self.dctSpreads = defaultdict(list)

lstMat = ['2Y','3Y','4Y','5Y','6Y','7Y','8Y','9Y','10Y','12Y','15Y','20Y','25Y','30Y']

lstSpreads = ['A','B','C','D','E']

def __GetSpreadMDValues__(self):
        lstSpreadRIC = []
        lstSpreadVal = []

        #Construct proper RICs to extract the data
        for i in range(0, len(self.lstSpreads)):
            for j in range(0, len(self.lstMat)):
                lstSpreadRIC.append(self.lstMat[j])
            #get market data
            lstSpreadVal = self.__GetListDataSnaps__('FIELD1', lstCSASpreadRIC)
            self.dctSpreads[self.lstSpreads[i]] = lstCSASpreadVal
            lstSpreadRIC = []
            lstSpreadVal = []

the problem is that self.dctSpreads (which is dictionary of list with floating values) increase memory usage but never release it after I done working with the class object. I have tried writing a simple destructor like calling method

def Destructor(self):
        gc.collect()

before finishing my run but that did not help.

Any thoughts are appreciated.

Upvotes: 1

Views: 1934

Answers (1)

kindall
kindall

Reputation: 184345

There's a difference between when Python will make unused memory available for use by your Python script and when it gets returned to the OS. The former happens immediately when there are no references to an object (assuming CPython) and the latter may not happen until the Python interpreter exits, depending on the type of objects allocated. This is normal and expected behavior.

Upvotes: 2

Related Questions