How can I decorate an instance method with a decorator class?

Question

Consider this small example:

import datetime as dt

class Timed(object):
    def __init__(self, f):
        self.func = f

    def __call__(self, *args, **kwargs):
        start = dt.datetime.now()
        ret = self.func(*args, **kwargs)
        time = dt.datetime.now() - start
        ret["time"] = time
        return ret

class Test(object):
    def __init__(self):
        super(Test, self).__init__()

    @Timed
    def decorated(self, *args, **kwargs):
        print(self)
        print(args)
        print(kwargs)
        return dict()

    def call_deco(self):
        self.decorated("Hello", world="World")

if __name__ == "__main__":
    t = Test()
    ret = t.call_deco()

which prints

Hello
()
{'world': 'World'}

Why is the self parameter (which should be the Test obj instance) not passed as first argument to the decorated function decorated?

If I do it manually, like :

def call_deco(self):
    self.decorated(self, "Hello", world="World")

it works as expected. But if I must know in advance if a function is decorated or not, it defeats the whole purpose of decorators. What is the pattern to go here, or do I misunderstood something?

Answer 1

You first have to understand how function become methods and how self is "automagically" injected.

Once you know that, the "problem" is obvious: you are decorating the decorated function with a Timed instance - IOW, Test.decorated is a Timed instance, not a function instance - and your Timed class does not mimick the function type's implementation of the descriptor protocol. What you want looks like this:

import types

class Timed(object):
    def __init__(self, f):
        self.func = f

    def __call__(self, *args, **kwargs):
        start = dt.datetime.now()
        ret = self.func(*args, **kwargs)
        time = dt.datetime.now() - start
        ret["time"] = time
        return ret

   def __get__(self, instance, cls):           
       return types.MethodType(self, instance)

Answer 2

I combined some answers and comments, especially from @PythonF, who had a google collab link that was pretty helpful in seeing how different methods actually work. My aim is not to be the best answer out there, because others know far better, but despite all of the other great answers, no one actually answered the question with code that is complete and reuseable, so here's some minimal code with test cases.

This can accept arguments and passes the instance in properly:

    class Decorator:
        def __init__(self, func = None, start_text = "Start", stop_text = "Stop"):
            self.func = func
            self.instance = None
            self.start_text = start_text
            self.stop_text = stop_text
    
        def __call__(self, func):
            if self.func is None:
                self.func = func
            def call(*args, **kwargs):
                if self.instance is None and len(args) > 0:
                    self.instance = args[0]
                # do stuff before
                print(f"--- {self.start_text} ---")
                wrapped_method = self.func(self.instance, *args[1:], **kwargs)
                # do stuff afterwards
                print(f"--- {self.stop_text} ---")
                return wrapped_method
            return call
    

    class HelloWorld:
        def __init__(self):
            self.test = "test"
    
        @Decorator(start_text="Starting...", stop_text="Done")
        def print(self, name):
            print(name)
            print(self.test)
            return 42


    hello_world = HelloWorld()
    hello_world.print("Max Musterman")

Answer 3

tl;dr

You can fix this problem by making the Timed class a descriptor and returning a partially applied function from __get__ which applies the Test object as one of the arguments, like this

class Timed(object):
    def __init__(self, f):
        self.func = f

    def __call__(self, *args, **kwargs):
        print(self)
        start = dt.datetime.now()
        ret = self.func(*args, **kwargs)
        time = dt.datetime.now() - start
        ret["time"] = time
        return ret

    def __get__(self, instance, owner):
        from functools import partial
        return partial(self.__call__, instance)

---

The actual problem

Quoting Python documentation for decorator,

The decorator syntax is merely syntactic sugar, the following two function definitions are semantically equivalent:

def f(...):
    ...
f = staticmethod(f)

@staticmethod
def f(...):
    ...

So, when you say,

@Timed
def decorated(self, *args, **kwargs):

it is actually

decorated = Timed(decorated)

only the function object is passed to the Timed, the object to which it is actually bound is not passed on along with it. So, when you invoke it like this

ret = self.func(*args, **kwargs)

self.func will refer to the unbound function object and it is invoked with Hello as the first argument. That is why self prints as Hello.

---

How can I fix this?

Since you have no reference to the Test instance in the Timed, the only way to do this would be to convert Timed as a descriptor class. Quoting the documentation, Invoking descriptors section,

In general, a descriptor is an object attribute with “binding behavior”, one whose attribute access has been overridden by methods in the descriptor protocol: __get__(), __set__(), and __delete__(). If any of those methods are defined for an object, it is said to be a descriptor.

The default behavior for attribute access is to get, set, or delete the attribute from an object’s dictionary. For instance, a.x has a lookup chain starting with a.__dict__['x'], then type(a).__dict__['x'], and continuing through the base classes of type(a) excluding metaclasses.

However, if the looked-up value is an object defining one of the descriptor methods, then Python may override the default behavior and invoke the descriptor method instead.

We can make Timed a descriptor, by simply defining a method like this

def __get__(self, instance, owner):
    ...

Here, self refers to the Timed object itself, instance refers to the actual object on which the attribute lookup is happening and owner refers to the class corresponding to the instance.

Now, when __call__ is invoked on Timed, the __get__ method will be invoked. Now, somehow, we need to pass the first argument as the instance of Test class (even before Hello). So, we create another partially applied function, whose first parameter will be the Test instance, like this

def __get__(self, instance, owner):
    from functools import partial
    return partial(self.__call__, instance)

Now, self.__call__ is a bound method (bound to Timed instance) and the second parameter to partial is the first argument to the self.__call__ call.

So, all these effectively translate like this

t.call_deco()
self.decorated("Hello", world="World")

Now self.decorated is actually Timed(decorated) (this will be referred as TimedObject from now on) object. Whenever we access it, the __get__ method defined in it will be invoked and it returns a partial function. You can confirm that like this

def call_deco(self):
    print(self.decorated)
    self.decorated("Hello", world="World")

would print

<functools.partial object at 0x7fecbc59ad60>
...

So,

self.decorated("Hello", world="World")

gets translated to

Timed.__get__(TimedObject, <Test obj>, Test.__class__)("Hello", world="World")

Since we return a partial function,

partial(TimedObject.__call__, <Test obj>)("Hello", world="World"))

which is actually

TimedObject.__call__(<Test obj>, 'Hello', world="World")

So, <Test obj> also becomes a part of *args, and when self.func is invoked, the first argument will be the <Test obj>.