Replicate inherance structure in Cython wrapper classes

Question

Let's say I have the following C++ code defined i AB.h:

class A {
public:
    void foo() {}
};

class B : public A {
public:
    void bar() {}
};

I want to wrap shared pointers to objects of these classes in Cython so I make the following pxd file:

from libcpp.memory cimport shared_ptr

cdef extern from "AB.h":
    cdef cppclass A:
        void foo()
    cdef cppclass B:
        void bar()

cdef class APy:
    cdef shared_ptr[A] c_self

cdef class BPy(APy): 
    cdef shared_ptr[B] c_self # <-- Error compiling Cython file: 'c_self' redeclared

And the following pyx file:

cdef class APy:
    def foo(self):
        return self.c_self.get().foo()

cdef class BPy(APy):
    def bar(self):
        return self.c_self.get().bar()

As you can see this does not compile. My goal is to have BPy inherit the foo python function from APy so that I don't have to write it twice. I can skip BPy(APy), and just write BPy, but then I have to write

def foo(self):
    return self.c_self.get().foo()

in the definition of BPy as well.

I can rename c_self in BPy to something else (e.g c_b_self) and then assign my pointer to both c_self and c_b_self when creating objects of BPy, but is there a more elegant way of achieving my goal?

Answer 1

It is surprisingly, that despite feeling naturally, there is no straight forward way to make PyB a subclass of PyA, - after all B is a subclass of A!

However, the desired hierarchy violates the Liskov substitution principle in some subtle ways. This principle says something along the lines:

If B is a subclass of A, then the objects of type A can be replaced by objects of type B without breaking the semantics of program.

It is not directly obvious, because the public interfaces of PyA and PyB are ok from Liskov's point of view, but there is one (implicit) property which makes our life harder:

• PyA can wrap any object of type A
• PyB can wrap any object of type B, also can do less than PyB!

This observation means there will be no beautiful solution for the problem, and your proposal of using different pointers isn't that bad.

My solution presented bellow has a very similar idea, only that I use a cast rather (which might improve the performance slightly by paying some type-safety), than to cache the pointer.

To make the example stand-alone I use inline-C-verbatim code and to make it more general I use classes without nullable constructors:

%%cython --cplus

cdef extern from *:
    """
    #include <iostream>
    class A {
    protected:
        int number;    
    public:
        A(int n):number(n){}
        void foo() {std::cout<<"foo "<<number<<std::endl;}
    };

    class B : public A {
    public:
        B(int n):A(n){}
        void bar() {std::cout<<"bar "<<number<<std::endl;}
    };
    """   
    cdef cppclass A:
        A(int n)
        void foo()
    cdef cppclass B(A): # make clear to Cython, that B inherits from A!
        B(int n)
        void bar()
 ...

Differences to your example:

1. constructors have a parameter and thus aren't nullable
2. I let the Cython know, that B is a subclass of A, i.e. use cdef cppclass B(A) - thus we can omit castings from B to A later on.

Here is the wrapper for class A:

...
cdef class PyA:
    cdef A* thisptr  # ptr in order to allow for classes without nullable constructors

    cdef void init_ptr(self, A* ptr):
        self.thisptr=ptr

    def __init__(self, n):
        self.init_ptr(new A(n))

    def __dealloc__(self):
        if NULL != self.thisptr:
            del self.thisptr

    def foo(self):
        self.thisptr.foo()
...

Noteworthy details are:

• thisptr is of type A * and not A, because A has no nullable constructor
• I use raw-pointer (thus __dealloc__ needed) for holding the reference, maybe one could considered using std::unique_ptr or std::shared_ptr, depending on how the class is used.
• When an object of class A is created, thisptr is automatically initialized to nullptr, so there is no need to explicitly set thisptr to nullptr in __cinit__ (which is the reason __cinit__ is omitted).
• Why __init__ and not __cinit__ is used will become evident in a little while.

And now the wrapper for class B:

...
cdef class PyB(PyA):
    def __init__(self, n):
        self.init_ptr(new B(n))

    cdef B* as_B(self):
        return <B*>(self.thisptr)  # I know for sure it is of type B*!

    def bar(self):
        self.as_B().bar()  

Noteworthy details:

• as_B is used to cast thisptr to B (which it really is) instead of keeping an cached B *-pointer.
• There is a subtle difference between __cinit__ and __init__: __cinit__ of the parent class will be always called, yet the __init__ of the parent class will only be called, when there is no implementation of the __init__-method for the class itself. Thus, we use __init__ because we would like to override/omit setting of self.thisptr of the basis-class.

And now (it prints to std::out and not the ipython-cell!):

>>> PyB(42).foo()
foo 42  
>>> PyB(42).bar()
bar 42

---

One last thought: I did the experience, that using inheritance in order to "save code" often led to problems, because one ended up with "wrong" hierarchies for wrong reasons. There might be another tools to reduce boilerplate code (like pybind11-framework mentioned by @chrisb) that are better for this job.

Answer 2

This isn't a direct answer to your question (would be curious if there is one!) - but one option would be to wrap with pybind11 - it can handle this without too much hassle.

wrapper.cpp

#include <pybind11/pybind11.h>

#include "AB.h"    

namespace py = pybind11;    
PYBIND11_MODULE(example, m) {
    py::class_<A>(m, "A")
        .def(py::init<>())
        .def("foo", &A::foo);

    py::class_<B, A>(m, "B") // second template param is parent
        .def(py::init<>())
        .def("bar", &B::bar);
}

setup.py

from setuptools import setup, Extension
import pybind11

setup(ext_modules=[Extension('example', ['wrapper.cpp'], 
                             include_dirs=[pybind11.get_include()])])