calling a Python function with variable number of input arguments from boost python

Question

Is it possible to call a python function that has a variable number of input arguments from inside boost::python:

In python I define a functions

def py_fn1(a):
    return 2 * a

def py_fn2(a, b):
    return a+b

def call_fn_py(fn, *args):
    return fn(*args)

a = call_fn_py(py_fn1, 15)  # returns 30
b = call_fn_py(py_fn2, 10, 11)  # returns 21

gives results as per the comments. I would like to implement call_fn(fn, *args) in C++ using boost python. So far I have:

using namespace boost::python;
object call_fn(object fn, tuple arg_in)
{
return fn(arg_in);
}


BOOST_PYTHON_MODULE(python_addins)
{
    using namespace boost::python;
def("call_fn", &call_fn);
}

but then doing something like:

import python_addins

d = python_addins.call_fn(py_fn1, (5,))  # returns (5,5)

which is not what I. am after. How can I write call_fn to behave like call_fn_py?

Thanks, Mark

Answer 1

It is possible to call a python function with a variable number of arguments. There are two approaches:

• Mimic variable number of arguments by creating a set of call_fn functions.
• Leverage python's ability to pack and unpack multiple objects into a tuple.

---

The first approach is fairly common for generic C++ programming. It requires creating a set of overloaded functions that vary on the amount of arguments: call_fn( fn ), call_fn( fn, A1 ), call_fn( fn, A1, ...An). The amount of boilerplate code can be reduced with C++11 variadic template functions. However, when defining the wrappers, the template functions exact type must be specified. Thus, this approach will have limits to the number of arguments based on what template instantiations are wrapped.

#include <boost/python.hpp>

using boost::python;
object call_fn1( object fn, object a1 )
{
  return fn( a1 );
}
object call_fn2( object fn, object a1, object a2 )
{
  return fn( a1, a2 );
}
object call_fn3( object fn, object a1, object a2, object a3 )
{
  return fn( a1, a2, a3 );
}

BOOST_PYTHON_MODULE(example)
{
  def( "call_fn", &call_fn1 );
  def( "call_fn", &call_fn2 );
  def( "call_fn", &call_fn3 );
}

And here is a demonstration:

>>> def py_fn1( a ): return 2 * a
... 
>>> def py_fn2( a, b ): return a + b
... 
>>> def call_fn_py( fn, *args ):
...     from example import call_fn
...     return call_fn( fn, *args )
... 
>>> call_fn_py( py_fn1, 15 ) 
30
>>> call_fn_py( py_fn2, 10, 11 )
21

---

The second approach leverages tuples. It require the caller to pack arguments into a tuple, and the calling function to unpack the arguments. However, since it is easier to do tuple packing and unpacking in python, example.call_fn can be patched in python so that the function argument is decorated with helper function that unpacks arguments before delegating.

In C++, create a call_fn function that accepts a single boost::python::tuple argument.

#include <boost/python.hpp>

using namespace boost::python;
object call_fn( object fn, tuple args )
{
  return fn( args );
}

BOOST_PYTHON_MODULE(example)
{
  def( "call_fn", &call_fn );
}

Now, create example_ext.py that will patch example.call_fn:

import example

def patch_call_fn():
    # Store handle to unpatched call_fn.
    original = example.call_fn

    # Helper function to create a closure that unpacks arguments
    # before delegating to the user function.
    def unpack_args( fn ):
        def wrapper( args ):
            return fn( *args )
        return wrapper

    # The patched function that will wrap the user function.
    def call_fn( fn, *args ):
        return original( unpack_args( fn ), args )

    return call_fn

# Patch example.
example.call_fn = call_fn = patch_call_fn()

The same demonstration can be used, and the only change required is that example_ext needs to be imported instead of example.

>>> def py_fn1( a ): return 2 * a
... 
>>> def py_fn2( a, b ): return a + b
... 
>>> def call_fn_py( fn, *args ):
...     from example_ext import call_fn
...     return call_fn( fn, *args )
... 
>>> call_fn_py( py_fn1, 15 ) 
30
>>> call_fn_py( py_fn2, 10, 11 )
21