CODEWITHSUNDEEP
pythonpython-3.xiterable-unpacking
Naitree
Naitree

Reputation: 1148

What happened internally during dict unpacking?

I'm trying to figure out what methods are called when a dict is unpacked, so that I can customize the process?

I suppose the following hack would have shown me the methods called during any method access of Dict instances:

class Dict(dict):
    def __getattribute__(self, name):
        print(name)
        return super().__getattribute__(name)

But the following session shows that no method is called to perform dict unpacking?

In [1]: d = Dict(a=1)
__class__
__class__

In [2]: {**d}
Out[2]: {'a': 1}

So what is actually happening here? How can I customize the unpacking process?

Edit

I don't think the question is a duplicate of the other one. Even implementing all the special methods mentioned in answers of that question, no of them are called during dict unpacking.

In [66]: class Dict(dict):
    ...:     def __getattribute__(self, name):
    ...:         print(name)
    ...:         return super().__getattribute__(name)
    ...:     def keys(self):
    ...:         print("hello")
    ...:         return super().keys()
    ...:     def __getitem__(self, key):
    ...:         print("hello")
    ...:         return super().__getitem__(key)
    ...:     def __len__(self):
    ...:         print("hello")
    ...:         return super().__len__()
    ...:     def __iter__(self):
    ...:         print("hello")
    ...:         return super().__iter__()
    ...:     

In [67]: d = Dict(a=1)
__class__
__class__

In [68]: {**d}
Out[68]: {'a': 1}

You can see no print line is called whatsoever. So my question remains unanswered.

FWIW, the python version is Python 3.6.5.

Upvotes: 1

Views: 54

Answers (1)

ShadowRanger
ShadowRanger

Reputation: 155438

This was a bug in how Python handled dict subclasses that was fixed in late September, 2018.

Before the fix, any dict subclass was converted to a normal dict using the concrete dict-specific C API methods (that bypass all dynamically defined overrides). After the fix, the code checks whether __iter__ (well, the C equivalent, tp_iter) has been overridden, and if so, it doesn't use the fast path for dicts. Checking __iter__ is slightly wrong IMO (the only two methods it actually uses are keys and __getitem__), but if you're overriding keys, you probably should be overriding __iter__ as well, so it's not a huge hassle to do so (in many cases, one can be an alias of the other, or at most a slim wrapper, depending on whether keys returns an iterator or a view object).

Given how recent the bugfix was, you'd need to upgrade Python to get the benefit. 3.6.7 and 3.7.1 are the first micro releases in their respective minor version lines that have the fix incorporated, so upgrading to either one should make your code work.

Upvotes: 1

Related Questions