CODEWITHSUNDEEP
pythonlistmemory
user16168
user16168

Reputation: 655

List memory usage

I am trying to improve the memory usage of my script in python, therefore I need to know what's RAM usage of my list. I measure the memory usage with

print str(sys.getsizeof(my_list)/1024/1024)

which hopefully would give me the size of the list in RAM in Mb.

it outputs 12 Mb, however in top command I see that my script uses 70% of RAM of 4G laptop when running.

In addition this list should contain a content from file of ~500Mb.

So 12Mb is unrealistic.

How can I measure the real memory usage?

Upvotes: 16

Views: 21873

Answers (2)

falsetru
falsetru

Reputation: 369044

sys.getsizeof only take account of the list itself, not items it contains.

According to sys.getsizeof documentation:

... Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to. ...

Use Pympler:

>>> import sys
>>> from pympler.asizeof import asizeof
>>>
>>> obj = [1, 2, (3, 4), 'text']
>>> sys.getsizeof(obj)
48
>>> asizeof(obj)
176

Note: The size is in bytes.

Upvotes: 33

Birdy40
Birdy40

Reputation: 367

Here is a code snippet that shows two suggested answers. However, the use of pympler gives what I believe to be the correct and more succinct answer to my question. Thank you falsetru :-) 

import sys
from pympler.asizeof import asizeof

tuple1 = ('1234','2019-04-27','23.4658')
tuple2 = ('1563','2019-04-27','19.2468')
klist1 = [tuple1]
klist2 = [tuple1,tuple2]
# The results for the following did not answer my question 
print ("sys.getsizeof(klist1): ",sys.getsizeof(klist1))
print ("sys.getsizeof(klist2): ",sys.getsizeof(klist2))    
# The results for the following give a quite reasonable answer
print ("asizeof(klist1): ",asizeof(klist1))
print ("asizeof(klist2): ",asizeof(klist2))

Upvotes: 0

Related Questions