CODEWITHSUNDEEP
python-3.xtracemalloc
mpr
mpr

Reputation: 43

Understanding tracemalloc output for small lists

I'm running the following simple script to get some familiarity with tracemalloc:

import tracemalloc
import linecache

def display_custom(snapshot):
   ...  # skipped for shorter code

tracemalloc.start()

a = [3.] * 512
b = [3.] * 513
c = [3.] * 514
d = [3.] * 515
e = [3.] * 516
f = [3.] * 517
g = [3.] * 518
h = [3.] * 519
i = [3.] * 520
j = [3.] * 521
k = [3.] * 522
l = [3.] * 523
m = [3.] * 524
n = [3.] * 525
o = [3.] * 526

snapshot = tracemalloc.take_snapshot()
display_custom(snapshot)

display_custom is just a slight modification of display_top from https://docs.python.org/3/library/tracemalloc.html. I get the following output:

Top 15 lines
#1 (Line 72): size=5264 B, count=2
    h = [3.] * 519
#2 (Line 79): size=4208 B, count=1
    o = [3.] * 526
#3 (Line 78): size=4200 B, count=1
    n = [3.] * 525
#4 (Line 77): size=4192 B, count=1
    m = [3.] * 524
#5 (Line 76): size=4184 B, count=1
    l = [3.] * 523
#6 (Line 75): size=4176 B, count=1
    k = [3.] * 522
#7 (Line 74): size=4168 B, count=1
    j = [3.] * 521
#8 (Line 73): size=4160 B, count=1
    i = [3.] * 520
#9 (Line 71): size=4144 B, count=1
    g = [3.] * 518
#10 (Line 70): size=4136 B, count=1
    f = [3.] * 517
#11 (Line 69): size=4128 B, count=1
    e = [3.] * 516
#12 (Line 68): size=4120 B, count=1
    d = [3.] * 515
#13 (Line 67): size=4112 B, count=1
    c = [3.] * 514
#14 (Line 66): size=4104 B, count=1
    b = [3.] * 513
#15 (Line 65): size=4096 B, count=1
    a = [3.] * 512
Total allocated size: 63392 B

What exactly happens with line 72? Why are two blocks used when other, larger lists only use one block? Why is the memory 5264 bytes? (for all entries with one block memory is just 8*number of elements)

Below is a shortened version of my actual code for reproducing. The only difference to my code above is that

Nothing else has changed so I wouldn't expect the memory allocations to change. The result similar to what I got before (but not identical).

import tracemalloc
import linecache


def display_custom(snapshot, key_type='lineno', limit=10):
    snapshot = snapshot.filter_traces((
        tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
        tracemalloc.Filter(False, "<unknown>"),
    ))
    top_stats = snapshot.statistics(key_type)

    print("Top %s lines" % limit)
    for index, stat in enumerate(top_stats[:limit], 1):
        frame = stat.traceback[0]
        print("#%s (Line %s): size=%d B, count=%d"
              % (index, frame.lineno, stat.size, stat.count))
        line = linecache.getline(frame.filename, frame.lineno).strip()
        if line:
            print('    %s' % line)

    other = top_stats[limit:]
    if other:
        size = sum(stat.size for stat in other)
        print("%s other: %d B" % (len(other), size))
    total = sum(stat.size for stat in top_stats)
    print("Total allocated size: %d B" % (total))


tracemalloc.start()

a = [3.] * 512
b = [3.] * 513
c = [3.] * 514
d = [3.] * 515
e = [3.] * 516
f = [3.] * 517
g = [3.] * 518
h = [3.] * 519
i = [3.] * 520
j = [3.] * 521
k = [3.] * 522
l = [3.] * 523
m = [3.] * 524
n = [3.] * 525
o = [3.] * 526

snapshot = tracemalloc.take_snapshot()
display_custom(snapshot, limit=15)

The output is:

Top 15 lines
#1 (Line 40): size=5280 B, count=2
    j = [3.] * 521
#2 (Line 45): size=4264 B, count=2
    o = [3.] * 526
#3 (Line 44): size=4256 B, count=2
    n = [3.] * 525
#4 (Line 43): size=4192 B, count=1
    m = [3.] * 524
#5 (Line 42): size=4184 B, count=1
    l = [3.] * 523
#6 (Line 41): size=4176 B, count=1
    k = [3.] * 522
#7 (Line 39): size=4160 B, count=1
    i = [3.] * 520
#8 (Line 38): size=4152 B, count=1
    h = [3.] * 519
#9 (Line 37): size=4144 B, count=1
    g = [3.] * 518
#10 (Line 36): size=4136 B, count=1
    f = [3.] * 517
#11 (Line 35): size=4128 B, count=1
    e = [3.] * 516
#12 (Line 34): size=4120 B, count=1
    d = [3.] * 515
#13 (Line 33): size=4112 B, count=1
    c = [3.] * 514
#14 (Line 32): size=4104 B, count=1
    b = [3.] * 513
#15 (Line 31): size=4096 B, count=1
    a = [3.] * 512
Total allocated size: 63504 B

Upvotes: 1

Views: 442

Answers (0)

Related Questions