Python decorator handles default arguments of the decorated function

Question

I want to create a 'cache' decorator for class methods, which registers in an internal class property the result of the method for avoiding to compute it several times (and I do not want to use a simple property, computed in the __init__, because I am not sure to compute it even once all the time).

The first idea is to create a decorator 'cache' similar to this:

def cache(func):
    name = "_{:s}".format(func.__name__)
    def wrapped(obj):
        if not hasattr(obj, name) or getattr(obj, name) is None:
            print "Computing..."
            setattr(obj, name, func(obj))
        else:
            print "Already computed!"
        return getattr(obj, name)
    return wrapped

class Test:
    @cache
    def hello(self):
        return 1000 ** 5

Everything works fine:

In [121]: t = Test()

In [122]: hasattr(t, '_hello')
Out[122]: False

In [123]: t.hello()
Computing...
Out[123]: 1000000000000000

In [124]: t.hello()
Already computed!
Out[124]: 1000000000000000

In [125]: hasattr(t, '_hello')
Out[125]: True

Now let us say that I want to do the same thing, but when the method can be called with arguments (keyworded and/or not). Of course, now we will store the results not in distinct properties (what would be the names?...), but in a dictionary, whose keys are composed with *args and **kwargs. Let us do it with tuples:

def cache(func):
    name = "_{:s}".format(func.__name__)
    def wrapped(obj, *args, **kwargs):
        if not hasattr(obj, name) or getattr(obj, name) is None:
            setattr(obj, name, {})
        o = getattr(obj, name)
        a = args + tuple(kwargs.items())
        if not a in o:
            print "Computing..."
            o[a] = func(obj, *args, **kwargs)
        else:
            print "Already computed!"
        return o[a]
    return wrapped

class Test:
    @cache
    def hello(self, *args, **kwargs):
        return 1000 * sum(args) * sum(kwargs.values())

In [137]: t = Test()

In [138]: hasattr(t, '_hello')
Out[138]: False

In [139]: t.hello()
Computing...
Out[139]: 0

In [140]: hasattr(t, '_hello')
Out[140]: True

In [141]: t.hello(3)
Computing...
Out[141]: 0

In [142]: t.hello(p=3)
Computing...
Out[142]: 0

In [143]: t.hello(4, y=23)
Computing...
Out[143]: 92000

In [144]: t._hello
Out[144]: {(): 0, (3,): 0, (4, ('y', 23)): 92000, (('p', 3),): 0}

Thanks to the fact that the method items turns a dictionary in a tuple without taking account on the order in the dictionary, it works perfectly if the keyworded arguments are not called in the same orders:

In [146]: t.hello(2, a=23,b=34)
Computing...
Out[146]: 114000

In [147]: t.hello(2, b=34, a=23)
Already computed!
Out[147]: 114000

Here is my problem: if the method has default arguments, then it doesn't work anymore:

class Test:
    @cache
    def hello(self, a=5):
        return 1000 * a

Now it does not work anymore:

In [155]: t = Test()

In [156]: t.hello()
Computing...
Out[156]: 5000

In [157]: t.hello(a=5)
Computing...
Out[157]: 5000

In [158]: t.hello(5)
Computing...
Out[158]: 5000

In [159]: t._hello
Out[159]: {(): 5000, (5,): 5000, (('a', 5),): 5000}

The result is computed 3 times, because the arguments are not given the same way (even if they are the "same" argument!).

Does someone know how I could catch the "default" values given to the function, inside the decorator?

Thank you

Answer 1

If you're using a sufficiently recent version of Python, you can use inspect.signature to get a Signature object that fully encapsulates the information about the function's arguments. Then you can call its bind method with the arguments your wrapper gets passed, to get a BoundArguments object. Call the apply_defaults method on the BoundArguments to fill in any missing arguments that have default values, and examine the arguments ordered dictionary to see an unambiguous listing of the parameters to the function and their values for this call:

import inspect

def cache(func):
    name = "_{:s}".format(func.__name__)
    sig = inspect.signature(func)
    def wrapped(obj, *args, **kwargs):
        cache_dict = getattr(obj, name, None)
        if cache_dict is None:
            cache_dict = {}
            setattr(obj, name, cache_dict)    
        bound_args = sig.bind(obj, *args, **kwargs)
        bound_args.apply_defaults()
        cache_key = tuple(bound_args.arguments.values())
        if not cache_key in cache_dict:
            print("Computing...")
            cache_dict[cache_key] = func(obj, *args, **kwargs)
        else:
            print("Already computed!")
        return cache_dict[cache_key]
    return wrapped

Note that I renamed your a and o variables to have more meaningful names. I also changed around the way the cache dictionary is set up on the object. Fewer getattr and setattr calls this way!

The inspect.signature function and associated types were added in Python 3.3, but the apply_defaults method on BoundArguments objects is new in Python 3.5. There's a backport of the basic functionality for older Python versions on PyPi, but it doesn't include apply_defaults yet, it seems. I'm going to report that as an issue on the backport's github tracker.

Answer 2

There can be various solutions depending on how complicated will be arguments' structure of function. The solution I prefer is to add inner function into hello. If you don't want to change the name of your cache, give it the same name your outer function have:

class Test:
    def hello(self, a=5):
        @cache
        def hello(self, a):
            return 1000 * a
        return hello(self, a)

t = Test()
t.hello()
t.hello(a=5)
t.hello(5)
t._hello

Out[111]: Computing...
Already computed!
Already computed!
{(5,): 5000}

Another approach is to add check for default variables in decorator, for example:

def cache(func):
    name = "_{:s}".format(func.__name__)
    def wrapped(obj, *args, **kwargs):
        if not hasattr(obj, name) or getattr(obj, name) is None:
            setattr(obj, name, {})
        o = getattr(obj, name)
        a = args + tuple(kwargs.items())
        if func.func_defaults: # checking if func have default variable
            for k in kwargs.keys():
                if k in func.func_code.co_varnames and kwargs[k] == func.func_defaults[0]:
                    a = ()
            if args:
                if args[0] == func.func_defaults[0]:
                    a = ()
        if not a in o:
            print "Computing..."
            o[a] = func(obj, *args, **kwargs)
        else:
            print "Already computed!"
        return o[a]
    return wrapped

class Test:
    @cache
    def hello(self, a=5):
        return 1000 * a

t = Test()
t.hello()
t.hello(a=5)
t.hello(5)
t._hello

Out[112]: Computing...
Already computed!
Already computed!
{(): 5000}

If you'd have, e.g. 2 default variables, first code (with inner function) still would work, whereas the second one would need changes in "default variable check rules".