CODEWITHSUNDEEP
pythonallocationcreation
Ian Goodfellow
Ian Goodfellow

Reputation: 2604

Tracking object allocation in python

Is there any method I can override that will allow me to use print statements / pdb / etc. to keep track of every time an instance of my class is allocated? While unpickling some objects I am seeming to get some that never have either __setstate__ or __init__ called on them. I tried overriding __new__ and printing out the id of every object I make in __new__, but I am still encountering objects with ids that were never printed.

Edit: here is my code I use for altering (instrumenting) __new__ of my class and all of its super-classes except for object itself:

class Allocator:
    def __init__(self, my_class):
       self.my_class = my_class
       self.old_new = my_class.__new__

    def new(self, * args, ** kargs):
        rval = self.old_new(*args, ** kargs)
        #rval = super(self.my_class,cls).__new__(cls)
        print 'Made '+str(self.my_class)+' with id '+str(id(rval))
        return rval

def replace_allocator(cls):
    if cls == object:
        return

    setattr(cls,'__new__',Allocator(cls).new)
    print cls.__base__

    try:
        for parent in cls.__base__:
            replace_allocator(parent)
   except:
        replace_allocator(cls.__base__)

I call replace_allocator on my classes' parent class as soon as it is imported in the main script. My class has a custom __new__ to begin with, which also prints out the id.

Upvotes: 10

Views: 1666

Answers (3)

Sven Marnach
Sven Marnach

Reputation: 601599

(This is more of a comment than an answer.)

Quoting Guido's Unifying types and classes in Python 2.2:

There are situations where a new instance is created without calling __init__ (for example when the instance is loaded from a pickle). There is no way to create a new instance without calling __new__ (although in some cases you can get away with calling a base class's __new__).

If you are using new-style classes (descendants of object), __new__() should always be called. I don't think the obscure cases "you can get away with calling a base class's __new__" in will happen accidently, though I don't know what these cases actually are.

And just to add an example:

In [1]: class A(object):
   ...:     def __new__(cls):    
   ...:         print "A"
   ...:         return object.__new__(cls)
   ...:     

In [2]: A()
A
Out[2]: <__main__.A object at 0xa3a95cc>

In [4]: object.__new__(A)
Out[4]: <__main__.A object at 0xa3a974c>

Upvotes: 5

s.m.
s.m.

Reputation: 8043

Not sure if this can help you, but Python's garbage collector has introspective capabilities that might be worth taking a look at.

Upvotes: 0

vz0
vz0

Reputation: 32923

Are you using new-style classes? For Pickle to call __setstate__, the __getstate__ method should also be defined on the class returning a non-False value.

Upvotes: 0

Related Questions