CODEWITHSUNDEEP
pythonjsonmemory-leaks
goldcode
goldcode

Reputation: 673

Does json.dumps leak?

import gc, json

class leak(object):
    def __init__(self):
        pass

gc.set_debug(gc.DEBUG_LEAK)
while True:
    leak_ = leak()
    json.dumps(leak_.__dict__, indent=True)
    gc.collect()
    print(f"garbage count: {len(gc.garbage)}")

Using the following code under Python 3.6.3, the garbage count keeps increasing and task manager records steady memory increase.

However without indent json.dumps(self.__dict__), no leak is observed.

Update: Code simplified to highlight the issue.

Upvotes: 3

Views: 1858

Answers (1)

goldcode
goldcode

Reputation: 673

There is no leak! ref: https://bugs.python.org/issue32045

indent=True just makes json to use Python implementation instead of C implementation. Python implementation uses closures which reference one other. Simple example not involving json is:

import gc

def f():
    def g():
        return h
    def h():
        return g
    return

gc.set_debug(gc.DEBUG_LEAK)
while True:
    f()
    gc.collect()
    print(f"garbage count: {len(gc.garbage)}")

The "leak" is caused by using gc.set_debug(gc.DEBUG_LEAK). gc.DEBUG_LEAK includes gc.DEBUG_COLLECTABLE, gc.DEBUG_UNCOLLECTABLE and gc.DEBUG_SAVEALL.

gc.DEBUG_SAVEALL causes garbage-collected objects to be saved in gc.garbage for inspection. In normal circumstances they are collected.

my reply:

you are right. i realized later i actually had a leak in a com instantiated object, assumed it was a leak in the python and tried to find it using the gc module.

The gc documentation led me down the garden path.

gc.garbage A list of objects which the collector found to be unreachable but could not be freed (uncollectable objects).

i assumed:

  • cyclic references are unreachable but can be freed and hence collectable.
  • __del__ finalizer (with cyclic references?) objects are unreachable and cannot be freed and hence uncollectable.

Upvotes: 2

Related Questions