CODEWITHSUNDEEP
pythonlistpython-3.xsizesys
user4302114
user4302114

Reputation:

Strange output sys.getsizeof()

I just ran these codes:

v = [1,2,'kite',100**100]

for x,y in enumerate(v):
    print ("{} size is: {}".format(y,sys.getsizeof(v[x])))
print ("Total size is: {} ".format(sys.getsizeof(v)))

Output:

1 size is: 14
2 size is: 14
kite size is: 29
100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 size is: 102
Total size is: 52 
>>>

The last element of v's size is 102, and total size is 52? Also even without the last element, still sum of the first 3 elements of the list is bigger than total size, my question is why? Python doing a zip process in lists ?

Another strange thing is, between this outputs; 

v = [""]

for x,y in enumerate(v):
    print ("{} size is: {}".format(y,sys.getsizeof(v[x])))
print ("Total size is: {} ".format(sys.getsizeof(v)))

v=[" "]
for x,y in enumerate(v):
    print ("{} size is: {}".format(y,sys.getsizeof(v[x])))
print ("Total size is: {} ".format(sys.getsizeof(v)))

Output:

>>> 
 size is: 27
Total size is: 40 
  size is: 26
Total size is: 40 
>>>

It is really strange, can anyone explain what's going on?

Upvotes: 2

Views: 2028

Answers (2)

paxdiablo
paxdiablo

Reputation: 882606

From the documentation (my bold) (a):

Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to.

So the size of v does not include the sizes of the elements it refers to.

If you change kite into kites, you'll also see that its size increases but not the size of v (I've replaced your big number with 100...00 in the output to ease formatting):

1 size is: 12
2 size is: 12
kite size is: 25
100...00 size is: 102
Total size is: 48

1 size is: 12
2 size is: 12
kites size is: 26
100...00 size is: 102
Total size is: 48

Think of it like this:

       /  +-----+
      | v | ref | -> 1
Size  |   | ref | -> 2
 of v |   | ref | -> 'kite'
      |   | ref | -> 100**100
       \  +-----+
                     \___________________________/
                      Size of things referred
                       to by v

(a) That page also has a link to a recipe for doing recursive size calculations if you need that information. The link is duplicated here for citation, and the code is duplicated below to make this answer more self-contained.

Plugging your structure into that code gives:

48 <type 'list'> [1, 2, 'kites', 100...00L]
12 <type 'int'> 1
12 <type 'int'> 2
26 <type 'str'> 'kites'
102 <type 'long'> 100...00L
200

The code, with your structure, is shown below.

from __future__ import print_function
from sys import getsizeof, stderr
from itertools import chain
from collections import deque
try:
    from reprlib import repr
except ImportError:
    pass

def total_size(o, handlers={}, verbose=False):
    """ Returns the approximate memory footprint an object and all of its contents.

    Automatically finds the contents of the following builtin containers and
    their subclasses:  tuple, list, deque, dict, set and frozenset.
    To search other containers, add handlers to iterate over their contents:

        handlers = {SomeContainerClass: iter,
                    OtherContainerClass: OtherContainerClass.get_elements}

    """
    dict_handler = lambda d: chain.from_iterable(d.items())
    all_handlers = {tuple: iter,
                    list: iter,
                    deque: iter,
                    dict: dict_handler,
                    set: iter,
                    frozenset: iter,
                   }
    all_handlers.update(handlers)     # user handlers take precedence
    seen = set()                      # track which object id's have already been seen
    default_size = getsizeof(0)       # estimate sizeof object without __sizeof__

    def sizeof(o):
        if id(o) in seen:       # do not double count the same object
            return 0
        seen.add(id(o))
        s = getsizeof(o, default_size)

        if verbose:
            print(s, type(o), repr(o), file=stderr)

        for typ, handler in all_handlers.items():
            if isinstance(o, typ):
                s += sum(map(sizeof, handler(o)))
                break
        return s

    return sizeof(o)


##### Example call #####

if __name__ == '__main__':
    v = [1,2,'kites',100**100]
    print(total_size(v, verbose=True))

Upvotes: 3

John Zwinck
John Zwinck

Reputation: 249592

This happens because your "Total size" is actually the size of the list structure without the contents. So you can store an object of any size there and it won't change your "Total size." You need a "recursive" getsizeof(), and for that, see here: Python deep getsizeof list with contents? or here: Deep version of sys.getsizeof

Upvotes: 1

Related Questions