Are there more than three types of methods in Python?

Question

I understand there are at least 3 kinds of methods in Python having different first arguments:

1. instance method - instance, i.e. self
2. class method - class, i.e. cls
3. static method - nothing

These classic methods are implemented in the Test class below including an usual method:

class Test():

    def __init__(self):
        pass

    def instance_mthd(self):
        print("Instance method.")

    @classmethod
    def class_mthd(cls):
        print("Class method.")

    @staticmethod
    def static_mthd():
        print("Static method.")

    def unknown_mthd():
        # No decoration --> instance method, but
        # No self (or cls) --> static method, so ... (?)
        print("Unknown method.")

In Python 3, the unknown_mthd can be called safely, yet it raises an error in Python 2:

>>> t = Test()

>>> # Python 3
>>> t.instance_mthd()
>>> Test.class_mthd()
>>> t.static_mthd()
>>> Test.unknown_mthd()

Instance method.
Class method.
Static method.
Unknown method.

>>> # Python 2
>>> Test.unknown_mthd()    
TypeError: unbound method unknown_mthd() must be called with Test instance as first argument (got nothing instead)

This error suggests such a method was not intended in Python 2. Perhaps its allowance now is due to the elimination of unbound methods in Python 3 (REF 001). Moreover, unknown_mthd does not accept args, and it can be bound to called by a class like a staticmethod, Test.unknown_mthd(). However, it is not an explicit staticmethod (no decorator).

Questions

1. Was making a method this way (without args while not explicitly decorated as staticmethods) intentional in Python 3's design? UPDATED
2. Among the classic method types, what type of method is unknown_mthd?
3. Why can unknown_mthd be called by the class without passing an argument?

---

Some preliminary inspection yields inconclusive results:

>>> # Types
>>> print("i", type(t.instance_mthd))
>>> print("c", type(Test.class_mthd))
>>> print("s", type(t.static_mthd))
>>> print("u", type(Test.unknown_mthd))                             
>>> print()

>>> # __dict__ Types, REF 002
>>> print("i", type(t.__class__.__dict__["instance_mthd"]))
>>> print("c", type(t.__class__.__dict__["class_mthd"]))
>>> print("s", type(t.__class__.__dict__["static_mthd"]))
>>> print("u", type(t.__class__.__dict__["unknown_mthd"]))          
>>> print()

i <class 'method'>
c <class 'method'>
s <class 'function'>
u <class 'function'>

i <class 'function'>
c <class 'classmethod'>
s <class 'staticmethod'>
u <class 'function'>

The first set of type inspections suggests unknown_mthd is something similar to a staticmethod. The second suggests it resembles an instance method. I'm not sure what this method is or why it should be used over the classic ones. I would appreciate some advice on how to inspect and understand it better. Thanks.

• REF 001: What's New in Python 3: “unbound methods” has been removed
• REF 002: How to distinguish an instance method, a class method, a static method or a function in Python 3?
• REF 003: What's the point of @staticmethod in Python?

Answer 1

Some background: In Python 2, "regular" instance methods could give rise to two kinds of method objects, depending on whether you accessed them via an instance or the class. If you did inst.meth (where inst is an instance of the class), you got a bound method object, which keeps track of which instance it is attached to, and passes it as self. If you did Class.meth (where Class is the class), you got an unbound method object, which had no fixed value of self, but still did a check to make sure a self of the appropriate class was passed when you called it.

In Python 3, unbound methods were removed. Doing Class.meth now just gives you the "plain" function object, with no argument checking at all.

Was making a method this way intentional in Python 3's design?

If you mean, was removal of unbound methods intentional, the answer is yes. You can see discussion from Guido on the mailing list. Basically it was decided that unbound methods add complexity for little gain.

Among the classic method types, what type of method is unknown_mthd?

It is an instance method, but a broken one. When you access it, a bound method object is created, but since it accepts no arguments, it's unable to accept the self argument and can't be successfully called.

Why can unknown_mthd be called by the class without passing an argument?

In Python 3, unbound methods were removed, so Test.unkown_mthd is just a plain function. No wrapping takes place to handle the self argument, so you can call it as a plain function that accepts no arguments. In Python 2, Test.unknown_mthd is an unbound method object, which has a check that enforces passing a self argument of the appropriate class; since, again, the method accepts no arguments, this check fails.

Answer 2

Are there more than three types of methods in Python?

Yes. There are the three built-in kinds that you mention (instance method, class method, static method), four if you count @property, and anyone can define new method types.

Once you understand the mechanism for doing this, it's easy to explain why unknown_mthd is callable from the class in Python 3.

A new kind of method

Suppose we wanted to create a new type of method, call it optionalselfmethod so that we could do something like this:

class Test(object):
    @optionalselfmethod
    def optionalself_mthd(self, *args):
        print('Optional-Self Method:', self, *args)

The usage is like this:

In [3]: Test.optionalself_mthd(1, 2)
Optional-Self Method: None 1 2

In [4]: x = Test()

In [5]: x.optionalself_mthd(1, 2)
Optional-Self Method: <test.Test object at 0x7fe80049d748> 1 2

In [6]: Test.instance_mthd(1, 2)
Instance method: 1 2

optionalselfmethod works like a normal instance method when called on an instance, but when called on the class, it always receives None for the first parameter. If it were a normal instance method, you would always have to pass an explicit value for the self parameter in order for it to work.

So how does this work? How you can you create a new method type like this?

The Descriptor Protocol

When Python looks up a field of an instance, i.e. when you do x.whatever, it check in several places. It checks the instance's __dict__ of course, but it also checks the __dict__ of the object's class, and base classes thereof. In the instance dict, Python is just looking for the value, so if x.__dict__['whatever'] exists, that's the value. However, in the class dict, Python is looking for an object which implements the Descriptor Protocol.

The Descriptor Protocol is how all three built-in kinds of methods work, it's how @property works, and it's how our special optionalselfmethod will work.

Basically, if the class dict has a value with the correct name¹, Python checks if it has an __get__ method, and calls it like type(x).whatever.__get__(x, type(x)) Then, the value returned from __get__ is used as the field value.

So for example, a trivial descriptor which always returns 3:

class GetExample:
    def __get__(self, instance, cls):
        print("__get__", instance, cls)
        return 3

class Test:
   get_test = GetExample()

Usage is like this:

In[22]: x = Test()

In[23]: x.get_test
__get__ <__main__.Test object at 0x7fe8003fc470> <class '__main__.Test'>
Out[23]: 3

Notice that the descriptor is called with both the instance and the class type. It can also be used on the class:

In [29]: Test.get_test
__get__ None <class '__main__.Test'>
Out[29]: 3

When a descriptor is used on a class rather than an instance, the __get__ method gets None for self, but still gets the class argument.

This allows a simple implementation of methods: functions simply implement the descriptor protocol. When you call __get__ on a function, it returns a bound method of instance. If the instance is None, it returns the original function. You can actually call __get__ yourself to see this:

In [30]: x = object()

In [31]: def test(self, *args):
    ...:     print(f'Not really a method: self<{self}>, args: {args}')
    ...:     

In [32]: test
Out[32]: <function __main__.test>

In [33]: test.__get__(None, object)
Out[33]: <function __main__.test>

In [34]: test.__get__(x, object)
Out[34]: <bound method test of <object object at 0x7fe7ff92d890>>

@classmethod and @staticmethod are similar. These decorators create proxy objects with __get__ methods which provide different binding. Class method's __get__ binds the method to the instance, and static method's __get__ doesn't bind to anything, even when called on an instance.

The Optional-Self Method Implementation

We can do something similar to create a new method which optionally binds to an instance.

import functools

class optionalselfmethod:

  def __init__(self, function):
    self.function = function
    functools.update_wrapper(self, function)

  def __get__(self, instance, cls):
    return boundoptionalselfmethod(self.function, instance)

class boundoptionalselfmethod:

  def __init__(self, function, instance):
    self.function = function
    self.instance = instance
    functools.update_wrapper(self, function)

  def __call__(self, *args, **kwargs):
    return self.function(self.instance, *args, **kwargs)

  def __repr__(self):
    return f'<bound optionalselfmethod {self.__name__} of {self.instance}>'

When you decorate a function with optionalselfmethod, the function is replaced with our proxy. This proxy saves the original method and supplies a __get__ method which returns a boudnoptionalselfmethod. When we create a boundoptionalselfmethod, we tell it both the function to call and the value to pass as self. Finally, calling the boundoptionalselfmethod calls the original function, but with the instance or None inserted into the first parameter.

Specific Questions

Was making a method this way (without args while not explicitly decorated as staticmethods) intentional in Python 3's design? UPDATED

I believe this was intentional; however the intent would have been to eliminate unbound methods. In both Python 2 and Python 3, def always creates a function (you can see this by checking a type's __dict__: even though Test.instance_mthd comes back as <unbound method Test.instance_mthd>, Test.__dict__['instance_mthd'] is still <function instance_mthd at 0x...>).

In Python 2, function's __get__ method always returns a instancemethod, even when accessed through the class. When accessed through an instance, the method is bound to that instance. When accessed through the class, the method is unbound, and includes a mechanism which checks that the first argument is an instance of the correct class.

In Python 3, function's __get__ method will return the original function unchanged when accessed through the class, and a method when accessed through the instance.

I don't know the exact rationale but I would guess that type-checking of the first argument to a class-level function was deemed unnecessary, maybe even harmful; Python allows duck-typing after all.

Among the classic method types, what type of method is unknown_mthd?

unknown_mthd is a plain function, just like any normal instance method. It only fails when called through the instance because when method.__call__ attempts to call the function unknown_mthd with the bound instance, it doesn't accept enough parameters to receive the instance argument.

Why can unknown_mthd be called by the class without passing an argument?

Because it's just a plain function, same as any other function. I just doesn't take enough arguments to work correctly when used as an instance method.

You may note that both classmethod and staticmethod work the same whether they're called through an instance or a class, whereas unknown_mthd will only work correctly when when called through the class and fail when called through an instance.

_{1. If a particular name has both a value in the instance dict and a descriptor in the class dict, which one is used depends on what kind of descriptor it is. If the descriptor only defines __get__, the value in the instance dict is used. If the descriptor also defines __set__, then it's a data-descriptor and the descriptor always wins. This is why you can assign over top of a method but not a @property; method only define __get__, so you can put things in the same-named slot in the instance dict, while @properties define __set__, so even if they're read-only, you'll never get a value from the instance __dict__ even if you've previously bypassed property lookup and stuck a value in the dict with e.g. x.__dict__['whatever'] = 3.}

Answer 3

@BrenBarn did a great job answering your question. This answer however, adds a plethora of details:

First of all, this change in bound and unbound method is version-specific, and it doesn't relate to new-style or classic classes:

2.X classic classes by default

>>> class A:
...     def meth(self): pass
... 
>>> A.meth
<unbound method A.meth>

>>> class A(object):
...     def meth(self): pass
... 
>>> A.meth
<unbound method A.meth>

3.X new-style classes by default

>>> class A:
...     def meth(self): pass
... 
>>> A.meth
<function A.meth at 0x7efd07ea0a60>

You've already mentioned this in your question, it doesn't hurt to mention it twice as a reminder.

---

>>> # Python 2
>>> Test.unknown_mthd()    
TypeError: unbound method unknown_mthd() must be called with Test instance as first argument (got nothing instead)

Moreover, unknown_mthd does not accept args, and it can be bound to a class like a staticmethod, Test.unknown_mthd(). However, it is not an explicit staticmethod (no decorator)

unknown_meth doesn't accept args, normally because you've defined the function without so that it does not take any parameter. Be careful and cautious, static methods as well as your coded unknown_meth method will not be magically bound to a class when you reference them through the class name (e.g, Test.unknown_meth). Under Python 3.X Test.unknow_meth returns a simple function object in 3.X, not a method bound to a class.

1 - Was making a method this way (without args while not explicitly decorated as staticmethods) intentional in Python 3's design? UPDATED

I cannot speak for CPython developers nor do I claim to be their representative, but from my experience as a Python programmer, it seems like they wanted to get rid of a bad restriction, especially given the fact that Python is extremely dynamic, not a language of restrictions; why would you test the type of objects passed to class methods and hence restrict the method to specific instances of classes? Type testing eliminates polymorphism. It would be decent if you just return a simple function when a method is fetched through the class which functionally behaves like a static method, you can think of unknown_meth to be static method under 3.X so long as you're careful not to fetch it through an instance of Test you're good to go.

2- Among the classic method types, what type of method is unknown_mthd?

Under 3.X:

>>> from types import *
>>> class Test:
...     def unknown_mthd(): pass
... 
>>> type(Test.unknown_mthd) is FunctionType 
True

It's simply a function in 3.X as you could see. Continuing the previous session under 2.X:

>>> type(Test.__dict__['unknown_mthd']) is FunctionType
True
>>> type(Test.unknown_mthd) is MethodType
True

unknown_mthd is a simple function that lives inside Test__dict__, really just a simple function which lives inside the namespace dictionary of Test. Then, when does it become an instance of MethodType? Well, it becomes an instance of MethodType when you fetch the method attribute either from the class itself which returns an unbound method or its instances which returns a bound method. In 3.X, Test.unknown_mthd is a simple function--instance of FunctionType, and Test().unknown_mthd is an instance of MethodType that retains the original instance of class Test and adds it as the first argument implicitly on function calls.

3- Why can unknown_mthd be called by the class without passing an argument?

Again, because Test.unknown_mthd is just a simple function under 3.X. Whereas in 2.X, unknown_mthd not a simple function and must be called be passed an instance of Test when called.