CODEWITHSUNDEEP
pythondictionarygenerator
Spack
Spack

Reputation: 494

Why do I get a KeyError in this generator?

I've got the following dictionary:

d = {
    'A': {
        'param': {
            '1': {
                'req': True,
            },
            '2': {
                'req': True,
            },
        },
    },
    'B': {
        'param': {
            '3': {
                'req': True,
            },
            '4': {
                'req': False,
            },
        },
    },
}

I want to have a generator which will give me for each first level keys, the required parameters.

req = {}
for key in d:
    req[key] = (p for p in d[key]['param'] if d[key]['param'][p].get('req', False))

So here, for each key in d, I get parameter p only if req is True.

However, when I try to use my generator, it raises a KeyError exception:

>>> req
{'A': <generator object <genexpr> at 0x27b8960>,
 'B': <generator object <genexpr> at 0x27b8910>}
>>> for elem in req['A']:
...     print elem
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-6-a96226f95cce> in <module>()
----> 1 for elem in req['A']:
      2     print elem
      3 

<ipython-input-4-1732088ccbdb> in <genexpr>((p,))
      1 for key in d:
----> 2         req[key] = (p for p in d[key]['param'] if d[key]['param'][p].get('req', False))
      3 

KeyError: '1'

Upvotes: 4

Views: 2284

Answers (2)

Kevin
Kevin

Reputation: 76194

The generator expressions you assign to req[key] binds on the key variable. But key changes from 'A' to 'B' in the loop. When you iterate over the first generator expression, it will evaluate key to 'B' in its if condition, even though key was 'A' when you created it.

The conventional way to bind to a variable's value and not its reference, is to wrap the expression in a lambda with a default value, and then call it immediately.

for key in d:
    req[key] = (lambda key=key: (p for p in d[key]['param'] if d[key]['param'][p].get('req', False)))()

Result:

1
2

Upvotes: 4

glglgl
glglgl

Reputation: 91017

This is because upon execution of the generator, the latest value of key is used.

Suppose the for key in d: iterates over the keys in the order 'A', 'B', the 1st generator is supposed to work with key = 'A', but due to closure issues, it uses the item with 'B' as key. And this has no '1' sub-entry.

Even worse, the key variable in the generator has two different values: the for p in d[key]['param'] part uses the "correct" value, while the if d[key]['param'][p].get('req', False) uses the "closure value", which is the last one.

Upvotes: 3

Related Questions