CODEWITHSUNDEEP
pythonclassgarbage-collectionclass-attributesclass-attribute
Might
Might

Reputation: 315

When are python classes and class attributes garbage collected?

class Member(object):
    def __init__(self, identifier):
        self.identifier = identifier
        print "Member __init__", self.identifier

    def __del__(self):
        print "Member __del__", self.identifier
        with open("/home/might/" + self.identifier, "w") as outF:
            outF.write(self.identifier)

class WithMembers(object):
    def __init__(self):
        print "WithMembers __init__"
        print WithMembers.classMem
        self.instanceMem = Member("instance mem")

    def __del__(self):
        print "WithMembers __del__"

    classMem = Member("class mem")

if __name__ == "__main__":
    print "main"
    WithMembers()
    #del WithMembers.classMem       # "Member __del__ class mem" before "end"
    #del WithMembers                # "Member __del__ class mem" after "end"
    print "end"

The above code is in Hidden.py and running python Hidden.py produces the following output:

Member __init__ class mem
main
WithMembers __init__
<__main__.Member object at 0x935aeec>
Member __init__ instance mem
WithMembers __del__
Member __del__ instance mem
end

I don't see Member __del__ class mem in output or the class mem file unless I un-comment one of the del statements. Why is this? When are python classes and class attributes garbage collected?

Upvotes: 6

Views: 4481

Answers (2)

Might
Might

Reputation: 315

This was reported as a bug in http://bugs.python.org/issue1545463 fixed in 3.4 but not backported (I was running 2.7). This was also explained in http://code.activestate.com/lists/python-list/504216/ . See below for output in python 3.5.

Based on above, my understanding is that in 2.7 , new style class WithMembers is still around (not cleaned up by GC) when interpreter exits. As a result, classMem is not garbage collected because WithMembers still references it.

Notice new style classes have cyclic references to themselves from __mro__ and some built-in descriptors (http://bugs.python.org/issue17950). Even though module-level new-style classes are considered dead by GC after module cleanup, the GC call to clean them up after module cleanup is disabled because it caused too many other problems.

This doesn't cause memory leak because OS cleans up resources after interpreter exits.

class Member(object):
    def __init__(self, identifier):
        self.identifier = identifier
        print("Member __init__ " + self.identifier)

    def __del__(self):
        print("Member __del__ " + self.identifier)
        with open("/home/might/" + self.identifier, "w") as outF:
            outF.write(self.identifier)

class WithMembers(object):
    def __init__(self):
        print("WithMembers __init__")
        print(WithMembers.classMem)
        self.instanceMem = Member("instance mem")

    def __del__(self):
        print("WithMembers __del__")

    classMem = Member("class mem")

if __name__ == "__main__":
    print("main")
    WithMembers()
    print("end")

outputs the following when run with python3 Hidden.py:

Member __init__ class mem
main
WithMembers __init__
<__main__.Member object at 0xb6fc8e2c>
Member __init__ instance mem
WithMembers __del__
Member __del__ instance mem
end
Member __del__ class mem
Exception ignored in: <bound method Member.__del__ of <__main__.Member object at 0xb6fc8e2c>>
Traceback (most recent call last):
  File "class_member_gc.py", line 8, in __del__
NameError: name 'open' is not defined

Upvotes: 5

Stephen Lin
Stephen Lin

Reputation: 4912

classMem is a class variable for class WithMembers which means it will be shared by all the instances of this class. It's a global thing in Python. That's why __del__ of class Member didn't get called when exiting the program.

Here comes a question: why Python doesn't simply set all reference counts to 0 when exiting the program so that all the __del__ functions could get called?

Like C++, which guarantees that destructors of global variables are called. The only way in Python to guarantee this is to go running around all modules and delete all their variables. But this means that __del__ method cannot trust that any global variables it might want to use still exist, since there is no way to know in what order variables are to be deleted.

There are two ways to force destruction of classMem.

One is del WithMembers.classMem. This will decrease count of references to 0 and __del__ will be automatically invoked.

The other is making class WithMembers an old-style class(not inheriting from object). Like this:

...

class WithMembers:
    def __init__(self):
        print "WithMembers __init__"
        print WithMembers.classMem
        self.instanceMem = Member("instance mem")
...

Output will be:

Member __init__ class mem
main
WithMembers __init__
<__main__.Member object at 0x00000000026C5278>
Member __init__ instance mem
WithMembers __del__
Member __del__ instance mem
end
Member __del__ class mem

Here is a very useful link to help you understand this answer better. http://www.electricmonk.nl/log/2008/07/07/python-destructor-and-garbage-collection-notes/

Hope it helps. :)

Upvotes: 0

Related Questions