How to make super() work in this non-ideal situation in python?

Question

class ClsOne(object):
    def __init__(self):
        super(ClsOne, self).__init__()
        print "Here's One"

class ClsTwo(ClsOne):
    def __init__(self):
        super(ClsTwo, self).__init__()
        print "Here's Two"

class ClsThree(ClsTwo): # Refer to one blackbox object
    def __init__(self):
        # super(ClsThree, self).__init__()
        print "Here's Three"

class ClsThreee(ClsTwo): # Refer to your custom object
    def __init__(self):
        super(ClsThreee, self).__init__()
        print "Here's Threee"

class ClsFour(ClsThree, ClsThreee): # Multiple Inheritance
    def __init__(self):
        super(ClsFour, self).__init__()
        print "Here's Four"

entity = ClsFour()

In this case, you are trying to combine ClsThree (which comes from a single compiled library and is very hard to change) and your own ClsThreee object together. Because ClsThree forgets to call super() in its constructor, their kid cannot execute ClsThreee's constructor when it uses super().

As a result, the output will just like this:

Here's Three
Here's Four

Obviously, I can manually call every bases of ClsFour rather than use super(), but it's a bit complicated when this problem scattered all over my codebase.

By the way, that blackbox stuff is PySide :)

SUPPLEMENT:

Thanks to @WillemVanOnsem and @RaymondHettinger, the previous ClsFour question is solved. But with some further investigations, I found the similar problem in PySide doesn't have same concept.

In ClsFour context, if you try to run:

print super(ClsFour, self).__init__

You'll get:

<bound method ClsFour.__init__ of <__main__.ClsFour object at 0x00000000031EC160>>

But in the following PySide context:

import sys
from PySide import QtGui

class MyObject(object):
    def __init__(self):
        super(MyObject, self).__init__()
        print "Here's MyObject"

class MyWidget(QtGui.QWidget, MyObject):
    def __init__(self):
        super(MyWidget, self).__init__()

app = QtGui.QApplication(sys.argv)
widget = MyWidget()
print super(MyWidget, widget).__init__

The result is:

<method-wrapper '__init__' of MyWidget object at 0x0000000005191D88>

It doesn't print "Here's MyObject" and the init attribute of super() has a different type as well. Previously, I try to simplify this problem as the ClsFour. But now I think it isn't totally the same.

I guess the problem occurs in shiboken library but I'm not sure.

TIPS:

The problem also appear in PyQt context, but you can make MyObject inherit from QObject to solve. This solution is useless in PySide.

Answer 1

Regarding the issue specific to PySide/PyQt4: one solution is to ensure that any mixins always preceed the Qt classes in the base-class definition:

import sys
from PySide import QtGui

class MyObject(object):
    def __init__(self, parent=None, other=None):
        super(MyObject, self).__init__(parent)
        print "Here's MyObject: %r" % other

class MyWidget(MyObject, QtGui.QWidget):
    def __init__(self, parent=None, other=None):
        super(MyWidget, self).__init__(parent, other)

app = QtGui.QApplication(sys.argv)
parent = QtGui.QWidget()
widget = MyWidget(parent, 'FOO')
print super(MyWidget, widget).__init__
# check that the QWidget.__init__ was called correctly
print 'widget.parent() is parent:', widget.parent() is parent

Output:

Here's MyObject: 'FOO'
<bound method MyWidget.__init__ of <__main__.MyWidget object at 0x7f53b92cca28>>
widget.parent() is parent: True

(NB: PyQt5 has improved Support for Cooperative Multi-inheritance, so this issue doesn't arise there).

Answer 2

super() is a proxy object that uses the Method Resolution Order (MRO) to determine what method to call when you perform a call on super().

If we inspect the __mro__ of ClassFour, we get:

>>> ClsFour.__mro__
(<class '__main__.ClsFour'>, <class '__main__.ClsThree'>, <class '__main__.ClsThreee'>, <class '__main__.ClsTwo'>, <class '__main__.ClsOne'>, <type 'object'>)

Or made it shorter myself (not the Python output):

>>> ClsFour.__mro__
(ClsFour, ClsThree, ClsThreee, ClsTwo, ClsOne, object)

Now super(T,self) is a proxy object that uses the MRO from (but excluding) T. So that means that super(ClsFour,self) is a proxy object that works with:

(ClsThree, ClsThreee, ClsTwo, ClsOne, object)  # super(ClsFour,self)

What will happen if you query an attribute (a method is also an attribute) of a class is that Python will walk through the MRO and inspect whether the element has such attribute. So it will first inspect whether ClsThree has an __init__ attribute, if not it will continue to look for it in ClsThreee and so on. From the moment it finds such attribute it will stop, and return it.

So super(ClsFour,self).__init__ will return the ClsThree.__init__ method. The MRO is also used to find methods, attributes, etc. that are not defined on the class level. So if you use self.x and x is not an attribute of the object nor of the ClsFour object, it will again walk through the MRO in search for x.

If you want to call the __init__ of all the direct parents of ClassFour you can use:

class ClsFour(ClsThree, ClsThreee):
    def __init__(self):
        # call *all* *direct* parents __init__
        for par in ClsFour.__bases__:
            par.__init__(self)

Which is probably the most elegant, since if the bases change, it will still work. Note that you have to make sure that __init__ exists for every parent. Since it is however defined at the object level, we can safely assume this. For other attributes however, we can not make that assumption.

EDIT: Mind that as a result super() does not necessary points to the parents, grandparents and/or ancestors of that class. But to parent classes of the object.

The super(ClsThree,self) in the ClsThree class, will - given it is an ClsFour object, work with the same mro (since it takes the mro from the self). So super(ClsThree,self) will inspect the following sequence of classes:

(ClsThreee, ClsTwo, ClsOne, object)

For instance, if we write (out of the scope of any class) super(ClsTwo,entity).__init__(), we get:

>>> super(ClsTwo,entity).__init__()
Here's One
>>> super(ClsThree,entity).__init__()
Here's Two
Here's Threee
>>> super(ClsThreee,entity).__init__()
Here's Two
>>> super(ClsFour,entity).__init__()
Here's Three

Answer 3

Summary

Because ClsThree forgets to call super() in its constructor, their kid cannot execute ClsThreee's constructor when it uses super().

This is covered in the "How to Incorporate a Non-cooperative Class" section of Super Considered Super blog post.

The key is to create an adapter class that plays by the rules to act cooperatively by calling super(). Use that adapter to wrap the original class.

Worked-out code

In the code below, AdaptThree is the new adapter class and ClsFour now inherits from AdaptThree instead of the original blackbox non-cooperative class.

class ClsOne(object):
    def __init__(self):
        print "Here's One"

class ClsTwo(ClsOne):
    def __init__(self):
        print "Here's Two"

class ClsThree(ClsTwo): # Refer to one blackbox object
    def __init__(self):
        # super(ClsThree, self).__init__()
        print "Here's Three"

class AdaptThree(object):
    def __init__(self):
        _three = ClsThree()
        super(AdaptThree, self).__init__()          

class ClsThreee(ClsTwo): # Refer to your custom object
    def __init__(self):
        super(ClsThreee, self).__init__()
        print "Here's Threee"

class ClsFour(AdaptThree, ClsThreee): # Multiple Inheritance
    def __init__(self):
        super(ClsFour, self).__init__()
        print "Here's Four"

entity = ClsFour()

This outputs:

Here's Three
Here's Two
Here's Threee
Here's Four

Other details

• In OP's example, ClsTwo also looks to be non-cooperative and should wrapped as well.

• Presumably these classes with have methods other than initialization. The adapter classes will need to wrap and dispatch those calls as well.

• Depending on the application, it may be easier to use composition rather than inheritance.