CODEWITHSUNDEEP
python
codebunny
codebunny

Reputation: 2899

Do Python dictionaries have all memory freed when reassigned?

Working in Python. I have a function that reads from a queue and creates a dictionary based on some of the XML tags in the record read from the queue, and returns this dictionary. I call this function in a loop forever. The dictionary gets reassigned each time. Does the memory previously used by the dictionary get freed at each reassignment, or does it get orphaned and eventually cause memory problems?

def readq():
    qtags = {}
    # Omitted code to read the queue record, get XML string, DOMify it

    qtags['result'] = "Success"
    qtags['call_offer_time'] = get_node_value_by_name(audio_dom, 'call_offer_time')
    # More omitted code to extract the rest of the tags

    return qtags

while signals.sigterm_caught == False:
    tags = readq()
    if tags['result'] == "Empty":
        time.sleep(SLEEP_TIME)
        continue
    # Do stuff with the tags

So when I reassign tags each time in that loop, will the memory used by the previous assignment get freed before being allocated by the new assignment?

Upvotes: 6

Views: 1413

Answers (3)

TheChes44
TheChes44

Reputation: 656

Usually Python can keep up with anything you throw at it. The Garbage collector used in Python uses reference counting, so your memory usage should be about constant, you won't see any spikes in memory. Right when you remove a reference (assign the variable to something else), the garbage collector throws the memory back into the "heap" if you will. So don't worry about memory. I have run simulators doing tests for hours rewriting variables, but the memory usage stays about the same. It will be freed when you assign it a new dictionary.

Upvotes: 1

agf
agf

Reputation: 176760

No, it will be freed AFTER the new object has been created.

In order for the reference count to go down on the old object, tags has to be pointed to the new object. This happens after readq returns, so at the very least both objects will exist from the beginning of qtags = {} to after tags = readq().

As @delnan stated, soon after tags has been pointed to the new object, the old one will be freed by the garbage collector as there is no longer a reference to it.

Upvotes: 2

user395760
user395760

Reputation:

The memory of an object will be freed if it can be proven (from the knowledge the language implementation has at runtime) that it cannot possibly be accessed any more and the garbage collector sees it fit to make a collection. That's the absolute minimum, and you shouldn't assume any more. And you usually shouldn't have to worry about anything more.

More practically speaking, it may be freed at some point in time between the last reference (where "reference" isn't limited to names in scope, but can be anything that makes the object reachable) being removed and memory running out. It doesn't have to be freed by the Python implementation running your code, it may as well leave the memory cleaning to the OS and forget about any finalizers and such. Note that there can be a noticeable delay between the last reference dying and memory usage actually dropping. But as mentioned before, most implementations go out of their way to avoid excessive memory usage if there is garbage to collect.

Even more practically, you'll propably be running this on CPython (the reference implementation), which always used and most propably will always use reference counting (augmented with a real GC to handle cyclic references), so unless there's a cyclic reference (relatively rare and your code doesn't look like it has them, but can occur e.g. in graph-like structures) it will be freed as soon as the last reference to it is deleted/overwritten. Of course, other implementations aren't that predictable - PyPy alone has half a dozen different garbage collectors, all but one falling under the above paragraph.

Upvotes: 10

Related Questions