Convert const wchar_t* to python string in boost python

Question

I've tried to figure out how to support const wchar_t* as a return type for a function exposed in boost python. My boost version is 1.52 and I'm working with python 2.7 if that makes any difference.

Somehow I can't get it to accept my conversion function. I've seen fragments of solutions to this problem on the internet but nothing that actually works or spells out how to do it right.

Here is my trivial non-working example:

#include <string>
#include <boost/python.hpp>
using namespace boost::python;


struct wchar_t_to_python_str
{
    static PyObject* convert(const wchar_t* )
    {
        std::string s = "I'm more interested in the function signature than how to do wide char to non-wide char conversion";
        return boost::python::incref(boost::python::object(s).ptr());
    }
};


void init_module()
{
     to_python_converter<const wchar_t*, wchar_t_to_python_str>();
}

const wchar_t* testWchar() {
    return L"Hello World";
}
const char* testChar() {
    return "Hello World";
}

BOOST_PYTHON_MODULE(test)
{
    // This works nicely, const char* is supported
    def("testChar", testChar);

    // This doesn't work, fails with this error 
    // 'awBoost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<T>'
    // def("testWchar", testWchar);

    // Throwing in a return value policy fires a compile time assert make_instance_impl
    // BOOST_MPL_ASSERT((mpl::or_<is_class<T>, is_union<T> >));
    // It seems like it gets confused by wchar_t not being a class, but it's hard to know
    def("testWchar", testWchar, return_value_policy<reference_existing_object>());
}

Answer 1

There are a few factors to this problem:

• The conversion functions enabled with to_python_converter are runtime conversions.
• Boost.Python does not provided builtin conversion support for wchar*. It may have been overlooked, as std::wstring support was added when Boost.Python was updated to support Python 3. The lack of builtin conversion support causes an internal class to require an appropriate CallPolicy during compilation. None of the provided ResultConverterGenerator models are candidates for this conversion, as they mainly affect object ownership/lifetime rather than type conversions.

There are two approaches that work within these constraints:

• Create a custom policy type to handle wchar* that fulfills the ResultConverterGenerator Concept.
• Wrap the function returning wchar* with a type that returns std::wstring.

Below is a complete example demonstrating both of these approaches:

#include <string>
#include <boost/function_types/parameter_types.hpp>
#include <boost/python.hpp>

/// @brief ResultConverterGenerator used to transform wchar_t to PyObject.
struct wchar_result_converter
{
  template <class T> struct apply
  {
    struct type
    {
      /// @brief Convert wchar_t to PyObject.
      PyObject* operator()(const wchar_t* str) const
      {
        // Using the Python/C API may be slighly cleaner.
        return PyUnicode_FromWideChar(str, wcslen(str));

        // Alternatively, Boost.Python's object type can be used.  While
        // Boost.Python does not know how to convert wchar_t to an object,
        // it does know how to convert std::wstring, so construct
        // a temporary to help in the conversion.
        //   return boost::python::incref(
        //     boost::python::object(std::wstring(str)).ptr());
      }

      /// @brief Used for documentation.
      const PyTypeObject* get_pytype() const { return 0; }
    }; // struct type
  };   // struct apply
}; 

/// @brief Modify the return type of a function using supplied CallPolicies.
template <typename ReturnType, typename Fn, typename Policy>
boost::python::object return_as(Fn fn, const Policy& policy)
{
  // Build MPL type representing signature of function, injecting the
  // explicitly provided return type.
  typedef typename boost::mpl::push_front<
    typename boost::function_types::parameter_types<Fn>::type,
    ReturnType
  >::type signature_type;

  return boost::python::make_function(fn, policy, signature_type());
}

/// @brief Modify the return type of a function using default_call_policies.
template <typename ReturnType, typename Fn>
boost::python::object return_as(Fn fn)
{
  return return_as<ReturnType>(fn, boost::python::default_call_policies());
}

// Test functions.
const char*    testChar()  { return  "Hello World"; }
const wchar_t* testWchar() { return L"Hello World"; }

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;

  // Expose non-wide test char support.
  python::def("testChar", testChar);

  // Expose wide char support by:
  // - providing a policy to convert the return value.
  // - manipulating the return type.
  python::def("testWchar1", &testWchar,
              python::return_value_policy<wchar_result_converter>());
  python::def("testWchar2", return_as<std::wstring>(&testWchar));
}

And its usage:

>>> import example
>>> a = example.testChar()
>>> print a
Hello World
>>> print type(a)
<type 'str'>
>>> b = example.testWchar1()
>>> print b
Hello World
>>> print type(b)
<type 'unicode'>
>>> c = example.testWchar2()
>>> print c
Hello World
>>> print type(c)
<type 'unicode'>