CODEWITHSUNDEEP
pythondecorator
Markus Grunwald
Markus Grunwald

Reputation: 333

Python: How can I ignore a special parameter in functools.lru_cache?

The function that I want to cache is something like:

def a(x, time='last'):

I have deterministic behaviour for every a(x,y), except when y=='last'. So when a(x, 'last') is called, I would like to call the "real thing" and an lru_cached function for everything else.

I imagine this could be possible with my own decorator:

def my_lru_cache(func):
    def function_wrapper(*args, **kwargs):
        if kwargs is not None:
            if 'time' in kwargs:
                return func(*args, **kwargs)
            else:
                return what?!?

    return function_wrapper

Am I completely wrong? How could this be done?

Upvotes: 2

Views: 3081

Answers (3)

mahmoudajawad
mahmoudajawad

Reputation: 105

Improving on @martijn-pieters answer for a use-case of mine; Here I created a wrapper that works on any function where kwarg skip_cache can be passed to specify whether to return cached value or calculate the value again.

from functools import lru_cache, wraps


def skippable_lru_cache(maxsize: int = 128, typed: bool = False):
    def wrapper_cache(func):
        cached_func = lru_cache(maxsize=maxsize, typed=typed)(func)

        @wraps(func)
        def wrapped_func(*args, **kwargs):
            if 'skip_cache' in kwargs and kwargs['skip_cache'] == True:
                # call the function directly
                return func(*args, **kwargs)
            else:
                # Remove skip_cache from kwargs so that its value doesn't affect stored results
                try:
                    del kwargs['skip_cache']
                except:
                    pass
                # use the lru_cache-wrapped version
                return cached_func(*args, **kwargs)

        wrapped_func.cache_info = cached_func.cache_info

        return wrapped_func

    return wrapper_cache


@skippable_lru_cache(maxsize=128)
def calc(v1: int, v2: int, skip_cache: bool = False):
    return v1 * v2


print('Run 1:')
for i in range(20):
    print(calc(i, 10))
print(calc.cache_info())

print('Run 2:')
for i in range(20):
    print(calc(i, 10, skip_cache=i > 9))
print(calc.cache_info())

Results:

Run 1:
0
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
160
170
180
190
CacheInfo(hits=0, misses=20, maxsize=128, currsize=20)
# All Run 1 calls are supposed to be hitting a miss as they are requesting cached value, but none are cached, generating a cache of size 20

Run 2:
0
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
160
170
180
190
CacheInfo(hits=10, misses=20, maxsize=128, currsize=20)
# For Run 2, only first 10 calls are requesting cache value, which shows in CacheInfo as hits=10 while still keeping the cache size at 20 although skip_cache is present with different value. For the second 10 calls, cache is completely being skipped resulting in the calls not registering as hit or miss, nor affecting the cache size

Upvotes: 0

Martijn Pieters
Martijn Pieters

Reputation: 1124388

Wrap the function in lru_cache(), then add your decorator on top and access the original uncached function via the __wrapped__ attribute, or better still, use the inspect.unwrap() function to strip the function of an arbitrary number of decorators:

from functools import wraps
from inspect import unwrap

def bypass_cache_last_time(func):
    @wraps(func)
    def function_wrapper(*args, **kwargs):
        if not 'time' in kwargs or kwargs['time'] == 'last':
            # Bypass any additional decorators and call function directly
            return unwrap(func)(*args, **kwargs)
        else:
            return func(*args, **kwargs)

        return function_wrapper

and use this as

@bypass_cache_last_time
@lru_cache()
def some_function(x, time='last'):
    # ...

The functools.wraps() decorator passes the ability to unwrap the decorator again forward, as it sets the __wrapped__ attribute on the wrapper.

Or make your decorator apply the lru_cache() decorator itself and retain your own copy of the original function when decorating:

def my_lru_cache(func):
    cached = lru_cache()(func)

    @wraps(func)
    def function_wrapper(*args, **kwargs):
        if not 'time' in kwargs or kwargs['time'] == 'last':
            # call the function directly
            return func(*args, **kwargs)
        else:
            # use the lru_cache-wrapped version
            return cached(*args, **kwargs)

    return function_wrapper

use this as

@my_lru_cache
def some_function(x, time='last'):
    # ...

Upvotes: 5

Eugene Yarmash
Eugene Yarmash

Reputation: 150101

You can call lru_cache() directly to get a 'wrapped' version of func using lru_cache(<args>)(func). Then you can return it from your wrapper:

def my_lru_cache(func):
    caching_func = lru_cache()(func)
    def function_wrapper(*args, **kwargs):        
        if kwargs.get('time') == 'last':
            return func(*args, **kwargs)
        return caching_func(*args, **kwargs)
    return function_wrapper

Upvotes: 1

Related Questions