CODEWITHSUNDEEP
pythonmemory-managementmemory-leaksmemory-profiling
Pietro Marchesi
Pietro Marchesi

Reputation: 882

Python memory_profiler: memory usages does not add up

The documentation for memory_profiler offers a simple example showing the changes in memory usage due to creating and deleting a list. 

from memory_profiler import profile

@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    return a

if __name__ == '__main__':
    my_func()

Which produces the following output:

Line #    Mem usage  Increment   Line Contents
==============================================
     3                           @profile
     4      5.97 MB    0.00 MB   def my_func():
     5     13.61 MB    7.64 MB       a = [1] * (10 ** 6)
     6    166.20 MB  152.59 MB       b = [2] * (2 * 10 ** 7)
     7     13.61 MB -152.59 MB       del b
     8     13.61 MB    0.00 MB       return a

When b is deleted, its memory usage is freed up. I was playing around and changed the example to the following, where I also delete a.

from memory_profiler import profile

@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del a
    del b
    return None

if __name__ == '__main__':
    my_func()

Here, however, deletion of a is not accompanied - as I would naively expect - by a decrease in memory usage. In fact, it seems as though nothing is happening. 

Line #    Mem usage    Increment   Line Contents
================================================
    12     25.1 MiB      0.0 MiB   @profile
    13                             def my_func():
    14     32.7 MiB      7.6 MiB       a = [1] * (10 ** 6)
    15    185.3 MiB    152.6 MiB       b = [2] * (2 * 10 ** 7)
    16    185.3 MiB      0.0 MiB       del a
    17     32.7 MiB   -152.6 MiB       del b
    18     32.7 MiB      0.0 MiB       return None

What is going on?

Upvotes: 1

Views: 2183

Answers (1)

Fabian Pedregosa
Fabian Pedregosa

Reputation: 6549

del a does in fact not delete the object, it merely marks the object as not used but it will be deleted later, whenever the garbage collector deems it appropriate. b is big enough to make the garbage collector delete it immediately, but apparently not so for a

Upvotes: 4

Related Questions