python property decorator for __name__ attr in class

Question

I find a good desc for python property in this link How does the @property decorator work in Python? below example shows how it works, while I find an exception for class attr 'name' now I have a reload function which will raise an error

@property
def foo(self): return self._foo
really means the same thing as

def foo(self): return self._foo
foo = property(foo)

here is my example

class A(object):
@property
def __name__(self):
    return 'dd'
    
a = A()
print(a.__name__)

dd this works, however below cannot work

class B(object):
    pass

def test(self):
    return 'test'

B.t = property(test)
print(B.t)
B.__name__ = property(test)

<property object at 0x7f71dc5e1180>
Traceback (most recent call last):
  File "<string>", line 23, in <module>
TypeError: can only assign string to B.__name__, not 'property'

Does anyone knows the difference for builtin name attr, it works if I use normal property decorator, while not works for the 2nd way. now I have a requirement to reload the function when code changes, however this error will block the reload procedure. Can anyone helps? thanks.

Answer 1

The short answer is: __name__ is deep magic in CPython.

So, first, let's get the technicalities out of the way. To quote what you said

@property
def foo(self): return self._foo
really means the same thing as

def foo(self): return self._foo
foo = property(foo)

This is correct. But it can be a bit misleading. You have this A class

class A(object):
  @property
  def __name__(self):
    return 'dd'

And you claim that it's equivalent to this B class

class B(object):
  pass

def test(self):
  return 'test'

B.__name__ = property(test)

which is not correct. It's actually equivalent to this

def test(self):
  return 'test'

class B(object):
  __name__ = property(test)

which works and does what you expect it to. And you're also correct that, for most names in Python, your B and my B would be the same. What difference does it make whether I'm assigning to a name inside the class or immediately after its declaration? Replace __name__ with ravioli in the above snippets and either will work. So what makes __name__ special?

That's where the magic comes in. When you define a name inside the class, you're working directly on the class' internal dictionary, so

class A:
  foo = 1

  def bar(self):
    return 1

This defines two things on the class A. One happens to be a number and the other happens to be a function (which will likely be called as a bound method). Now we can access these.

A.foo # Returns 1, simple access
A.bar # Returns the function object bar
A().foo # Returns 1
A().bar # Returns a bound method object

When we look up the names directly on A, we simply access the slots like we would on any object. However, when we look them up on A() (an instance of A), a multi-step process happens

1. Look up the name on the instance's __dict__ directly.
2. If that failed, then look up the name on the class' __dict__.
3. If we found it on the class, see if there's a __get__ on the result and call it.

That third step is what allows bound method objects to work, and it's also the mechanism underlying the property decorators in Python.

Let's go through this whole process with a property called ravioli. No magic here.

class A(object):
  @property
  def ravioli(self):
    return 'dd'

When we do A().ravioli, first we see if there's a ravioli on the instance we just made. There isn't, so we check the class' __dict__, and indeed we find a property object at that position. That property object has a __get__, so we call it, and it returns 'dd', so indeed we get the string 'dd'.

>>> A().ravioli
'dd'

Now I would expect that, if I do A.ravioli, we will simply get the property object. Since we're not calling it on an instance, we don't call __get__.

>>> A.ravioli
<property object at 0x7f5bd3690770>

And indeed, we get the property object, as expected.

Now let's do the exact same thing but replace ravioli with __name__.

class A(object):
  @property
  def __name__(self):
    return 'dd'

Great! Now let's make an instance.

>>> A().__name__
'dd'

Sensible, we looked up __name__ on A's __dict__ and found a property, so we called its __get__. Nothing weird.

Now

>>> A.__name__
'A'

Um... what? If we had just found the property on A's __dict__, then we should see that property here, right?

Well, no, not always. See, in the abstract, foo.bar normally looks in foo.__dict__ for a field called bar. But it doesn't do that if the type of foo defines a __getattribute__. If it defines that, then that method is always called instead.

Now, the type of A is type, the type of all Python types. Read that sentence a few times and make sure it makes sense. And if we do a bit of spelunking into the CPython source code, we see that type actually defines __getattribute__ and __setattr__ for the following names:

• __name__
• __qualname__
• __bases__
• __module__
• __abstractmethods__
• __dict__
• __doc__
• __text_signature__
• __annotations__

That explains how __name__ can serve double duty as a property on the class instances and also as an accessible field on the same class. It also explains why you get that highly specialized error message when reassigning to B.__name__: the line

B.__name__ = property(test)

is actually equivalent to

type.__setattr__(B, '__name__', property(test))

which is calling our special-case checker in CPython.

For any other type in Python, in particular for user-defined types, we could get around this with object.__setattr__. Unfortunately,

>>> object.__setattr__(B, '__name__', property(test))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't apply this __setattr__ to type object

There's a really specific check to make sure we don't do exactly this, and the comment reads

/* Reject calls that jump over intermediate C-level overrides. */

We also can't use metaclasses to override __setattr__ and __getattribute__, because the instance lookup procedure specifically doesn't call those (in the above examples, __getattribute__ was called in every case except the one we care about for property purposes). I even tried subclassing str to trick __setattr__ into accepting our made-up value

class NameProperty(str):

    def __new__(cls, value, **kwargs):
        return str.__new__(cls, value)

    def __init__(self, value, method):
        self.method = method

    def __get__(self, instance, owner):
        return self.method(instance)

B.__name__ = NameProperty(B.__name__, method=test)

This actually passes the __setattr__ check, but it doesn't assign to B.__dict__ (since the __setattr__ still assigns to the actual CPython-level name, not to B.__dict__['__name__']), so the property lookup doesn't work.

So... that's how I reached my conclusion of: __name__ is deep magic in CPython. All of the usual Python metaprogramming techniques have failed, and all of the methods getting called are written deep down in C. My advice to you is: Stop using __name__ for things it's not intended for, or be prepared to write some C code and hack on CPython directly.