CODEWITHSUNDEEP
pythonc++cython
Jon Petter
Jon Petter

Reputation: 97

Cython: std::function callbacks with custom parameter types

Please read this post before answering: Pass a closure from Cython to C++

In the accepted answers, it is neatly shown how a python function is converted into a std::function using Boost Python.

Following this example I'm able to wrap functions taking an std::function as an argument and call them using a python function as input. However, this only works when the std::function parameters are primitives like int, double, string etc.

Any guidance on how to make this work for custom types as well will be highly appreciated.

Upvotes: 2

Views: 1125

Answers (1)

DavidW
DavidW

Reputation: 30888

This won't be a complete answer - it assumes you can fill in the gaps from my previous answer that the question was based on. Unfortunately it is a little bit more complicated than that case.

Just to define the problem - assume you have a parameter of a custom C++ class, like:

class cpp_class {
  // some non-trivial contents
};

and thus your C++ interface looks like this:

void call_some_std_func(std::function<void(cpp_class&)> callback) {
    callback(5,std::string("hello"));
}

The first thing to do is to write a Cython wrapper for your C++ class (in principle you could make a Boost Python wrapper instead). Here you need to make a choice of about "ownership" of the C++ object. The first choice is to make a copy:

cdef extern from "cpp_file.hpp":
  cppclass cpp_class:
    pass # details

cdef class CyWrapper:
  cdef cpp_class* ptr
  def __dealloc__(self):
    del self.ptr
  # other details following standard wrapper pattern

cdef public make_CyWrapper(cpp_class& x):
  obj = CyWrapper()
  obj.ptr = new cpp_class(x)
  return obj

I've created a wrapper class with a destructor that handles the memory and a publicly accessible constructor function that can be called from external code. This version is safe because your wrapper owns the object it holds and so there can be no writes to invalid memory. However, because it makes a copy, you can't make changes to the original C++ object.

A second option is to hold a pointer to an object you don't own. The code is basically identical except you remove the __dealloc__ and avoid making a copy in make_CyWrapper:

obj.ptr = &x // instead of new cpp_class(x)

This is unsafe - you need to ensure your C++ object outlives the Cython wrapper - but allows you to modify the object.

You could also imagine a few other options: you could take ownership of an existing object with your Cython wrapper (Such a scheme would have to pass by pointer rather than reference, or it could use move constructors); you could deconstruct your C++ class into a representation expressed in basic types and pass those to Python; you could use shared pointers to split the ownership; or you have a more elaborate way of marking your Cython wrapper as "invalid" once your held C++ instance is destructed.

What you do next depends on whether you're using Boost Python (for it's convenient, callable wrapping of Python objects) or if you're making your own version. (I showed both possibilities in the previous answer).

Assuming Boost Python, you need to do two things - tell it about the conversion and make sure that it imports the module that your wrapper is defined in (if you don't do this you get exciting segmentation faults)

struct convert_to_PyWrapper {
 static PyObject* convert(const cpp_class& rhs) {
    // the const_cast here is a bit dodgy, but was needed to make it work
    return make_CyWrapper(const_cast<cpp_class&>(rhs));
}
};

inline void setup_boost_python() {
    PyInit_your_module_name(); // named inityour_module_name in Python 2
    boost::python::to_python_converter<
        cpp_class,
        convert_to_PyWrapper>();
}

You need to make sure that your Python/Cython code calls "setup_boost_python" before attempting to use the callback (if you put it at module level it's done on import, which is ideal).

If you're following my "manual" scheme (avoiding the dependency on Boost Python) then you need to modify the call_obj Cython function that does the C++ to Cython type conversion.

cdef public void call_obj(obj, cpp_class& c):
    obj(make_CyWrapper(c))

You also need to ensure the wrapper Cython module is imported before use (otherwise you get segmentation faults). I did this in "py_object_wrapper.hpp" but providing it's done once somewhere you can place it where you like.

void operator()(cpp_class& a) {
    PyInit_your_module_name();
    if (held) {
        call_obj(held,a);
    }
}

Upvotes: 1

Related Questions