CODEWITHSUNDEEP
pythonlazy-evaluation
kjo
kjo

Reputation: 35321

How to implement a lazy setdefault?

One minor annoyance with dict.setdefault is that it always evaluates its second argument (when given, of course), even when the first argument is already a key in the dictionary.

For example:

import random
def noisy_default():
    ret = random.randint(0, 10000000)
    print 'noisy_default: returning %d' % ret
    return ret

d = dict()
print d.setdefault(1, noisy_default())
print d.setdefault(1, noisy_default())

This produces ouptut like the following:

noisy_default: returning 4063267
4063267
noisy_default: returning 628989
4063267

As the last line confirms, the second execution of noisy_default is unnecessary, since by this point the key 1 is already present in d (with value 4063267).

Is it possible to implement a subclass of dict whose setdefault method evaluates its second argument lazily?

EDIT:

Below is an implementation inspired by BrenBarn's comment and Pavel Anossov's answer. While at it, I went ahead and implemented a lazy version of get as well, since the underlying idea is essentially the same.

class LazyDict(dict):
    def get(self, key, thunk=None):
        return (self[key] if key in self else
                thunk() if callable(thunk) else
                thunk)


    def setdefault(self, key, thunk=None):
        return (self[key] if key in self else
                dict.setdefault(self, key,
                                thunk() if callable(thunk) else
                                thunk))

Now, the snippet

d = LazyDict()
print d.setdefault(1, noisy_default)
print d.setdefault(1, noisy_default)

produces output like this:

noisy_default: returning 5025427
5025427
5025427

Notice that the second argument to d.setdefault above is now a callable, not a function call.

When the second argument to LazyDict.get or LazyDict.setdefault is not a callable, they behave the same way as the corresponding dict methods.

If one wants to pass a callable as the default value itself (i.e., not meant to be called), or if the callable to be called requires arguments, prepend lambda: to the appropriate argument. E.g.:

d1.setdefault('div', lambda: div_callback)

d2.setdefault('foo', lambda: bar('frobozz'))

Those who don't like the idea of overriding get and setdefault, and/or the resulting need to test for callability, etc., can use this version instead:

class LazyButHonestDict(dict):
    def lazyget(self, key, thunk=lambda: None):
        return self[key] if key in self else thunk()


    def lazysetdefault(self, key, thunk=lambda: None):
        return (self[key] if key in self else
                self.setdefault(key, thunk()))

Upvotes: 46

Views: 8220

Answers (5)

WloHu
WloHu

Reputation: 1527

For Python 3.8+ the best I can come with is a function.

from typing import MutableMapping, Callable, TypeVar

K = TypeVar('K')
V = TypeVar('V')
MISSING = object()


def setdefault_lazy(d: MutableMapping[K ,V], key: K, func: Callable[[], V]) -> V:
    if (value := d.get(key, MISSING)) is MISSING:
        d[key] = value = func()
    return value

Use case:

d = dict()
print(setdefault_lazy(d, 1, noisy_default))
print(setdefault_lazy(d, 1, noisy_default))

You can make your own dict class which uses this function like so:

class MyDict(dict):
    setdefault_lazy = setdefault_lazy

d = MyDict(name="John")
print(d.setdefault_lazy('age', noisy_default))
print(d.setdefault_lazy('age', noisy_default))

Upvotes: 1

Dennis Golomazov
Dennis Golomazov

Reputation: 17339

There seems to be no one-liner that doesn't require an extra class or extra lookups. For the record, here is a easy (even not concise) way of achieving that without either of them.

try:
    value = dct[key]
except KeyError:
    value = noisy_default()
    dct[key] = value
return value

Upvotes: 2

Cesar Canassa
Cesar Canassa

Reputation: 20183

You can do that in a one-liner using a ternary operator:

value = cache[key] if key in cache else cache.setdefault(key, func(key))

If you are sure that the cache will never store falsy values, you can simplify it a little bit:

value = cache.get(key) or cache.setdefault(key, func(key))

Upvotes: 16

Santtu Pajukanta
Santtu Pajukanta

Reputation: 1381

This can be accomplished with defaultdict, too. It is instantiated with a callable which is then called when a nonexisting element is accessed.

from collections import defaultdict

d = defaultdict(noisy_default)
d[1] # noise
d[1] # no noise

The caveat with defaultdict is that the callable gets no arguments, so you can not derive the default value from the key as you could with dict.setdefault. This can be mitigated by overriding __missing__ in a subclass:

from collections import defaultdict

class defaultdict2(defaultdict):
    def __missing__(self, key):
        value = self.default_factory(key)
        self[key] = value
        return value

def noisy_default_with_key(key):
    print key
    return key + 1

d = defaultdict2(noisy_default_with_key)
d[1] # prints 1, sets 2, returns 2
d[1] # does not print anything, does not set anything, returns 2

For more information, see the collections module.

Upvotes: 26

Pavel Anossov
Pavel Anossov

Reputation: 62928

No, evaluation of arguments happens before the call. You can implement a setdefault-like function that takes a callable as its second argument and calls it only if it is needed.

Upvotes: 12

Related Questions