CODEWITHSUNDEEP
pythonoop
Yu Chen
Yu Chen

Reputation: 7460

How does object.__getattribute__ redirect to __get__ method on my descriptor and __getattr__?

I have a two-part question regarding the implementation of object.__getattribute(self, key), but they are both centered around my confusion of how it is working.

I've defined a data descriptor called NonNullStringDescriptor that I intended to attach to attributes.

class NonNullStringDescriptor:

    def __init__(self, value: str = "Default Name"):
        self.value = value

    def __get__(self, instance, owner):
        return self.value

    def __set__(self, instance, value):
        if isinstance(value, str) and len(value.strip()) > 0:
            self.value = value
        else:
            raise TypeError("The value provided is not a non-null string.")

I then declare a Person class with an attribute of name.

class Person:
    name = NonNullStringDescriptor()

    def __init__(self):
        self.age = 22

    def __getattribute__(self, key):
        print(f"__getattribute__({key})")
        v = super(Person, self).__getattribute__(key)
        print(f"Returned {v}")

        if hasattr(v, '__get__'):
            print("Invoking __get__")
            return v.__get__(None, self)
        return v

    def __getattr__(self, item):
        print(f"__getattr__ invoked.")
        return "Unknown"

Now, I try accessing variable attributes, some that are descriptors, some normal instance attributes, and others that don't exist:

    person = Person()
    print("Printing", person.age) # "normal" attribute
    print("Printing", person.hobby) # non-existent attribute
    print("Printing", person.name) # descriptor attribute

The output that is see is 

__getattribute__(age)
Returned 22
Printing 22
__getattribute__(hobby)
__getattr__ invoked.
Printing Unknown
__getattribute__(name)
Returned Default Name
Printing Default Name

I have two main questions, both of which center around super(Person, self).__getattribute__(key):

  1. When I attempt to access a non-existent attribute, like hobby, I see that it redirects to __getattr__, which I know is often the "fallback" method in attribute lookup. However, I see that __getattribute__ is what is invoking this method. However, the Returned ... console output is never printed meaning that the rest of the __getattribute__ does not complete - so how exactly is __getattribute__ invoking __getattr__ directly, returning this default "Unknown" value without executing the rest of its own function call?

  2. I would expect that what is returned from super(Person, self).__getattribute__(key) (v), is the data descriptor instance of NonNullStringDescriptor. However, I see that v is actually the string "Default Name" itself! So how does object.__getattribute__(self, key) just know to use the __get__ method of my descriptor, instead of returning the descriptor instance?

There's references to behavior in the Descriptor Protocol:

If the looked-up value is an object defining one of the descriptor methods, then Python may override the default behavior and invoke the descriptor method instead.

But it's never explicitly defined to me what is actually happening in object.__getattribute(self, key) that performs the override. I know that ultimately person.name gets converted into a low-level call to type(person).__dict__["name"].__get__(person, type(person))- is this all happening in object.__getattribute__?

I found this SO post, which describes proper implementation of __getattribute__, but I'm more curious at what is actually happening in object.__getattribute__. However, my IDE (PyCharm) only provides a stub for its implementation:

def __getattribute__(self, *args, **kwargs): # real signature unknown
        """ Return getattr(self, name). """
        pass

Upvotes: 1

Views: 1316

Answers (1)

user2357112
user2357112

Reputation: 280456

__getattribute__ doesn't call __getattr__. The __getattr__ fallback happens in the attribute access machinery, after __getattribute__ raises an AttributeError. If you want to see the implementation, it's in slot_tp_getattr_hook in Objects/typeobject.c.

object.__getattribute__ knows to call __get__ because there's code in object.__getattribute__ that calls __get__. It's pretty straightforward. If you want to see the implementation, object.__getattribute__ is PyObject_GenericGetAttr in the implementation (yes, even though it says GetAttr - the C side of things is a little different from the Python side), and there are two __get__ call sites (one for data descriptors and one for non-data descriptors), here and here.

Upvotes: 5

Related Questions