CODEWITHSUNDEEP
pythonpython-3.xtype-conversioncythonwrapper
Romzie
Romzie

Reputation: 457

Cython with Python3.5: convert list of string to char**

I am trying to pass a list of string to a C++ function that takes a char** as argument using Cython.

I tried other solutions I can't remember but I mainly tried the two following options:

  1. Using a convertion function extracted from here, which is basically the same answer than here, but it raises an error, on the g++ compilation which is

    cy_wrapper.cpp: In function 'char** _pyx_f_10cy_wrapper_to_cstring_array(PyObject*)':
cy_wrapper.cpp:1223:44: error 'PyString_AsString' was not declared in this scope
    __pyx_t_5 = PyString_AsString(__pyx_t_4); if (unlikely(__pyx_t_5 == ((char *)NULL))

  2. Using the encoding method of python strings, as suggested in here but the answer is not satisfying since it raises the error:

    cy_wrapper.pyx:14:35: Storing unsafe C derivative of temporary Python reference

I looked in the library files at ~/.local/lib/python3.5/site-packages/Cython/Includes/cpython and I found the file string.pxd which contains the function PyString_AsString I need.

Why is it not found? If there is no possibility to use it, is there a workaround?

I am working on Ubuntu 16.04.4 LTS (tegra kernel) on a arm64 arch.

My cy_wrapper.pyx is exactly like this:

from cpython.string cimport PyString_AsString
from libc.stdlib cimport malloc

cdef extern:
    cdef cppclass imageNet:
        imageNet* Create(int argc, char** argv)

cdef char** to_cstring_array(list_str):
    cdef char** ret = <char **>malloc(len(list_str) * sizeof(char *))
    for i in range(len(list_str)):
        ret[i] = PyString_AsString(list_str[i])
    return ret

cdef class PyImageNet:
    cdef imageNet* c_net
    def Create(self, argc, kwargs):
        cdef char** c_argv = to_cstring_array(kwargs)
        return PyImageNetFactory(self.c_net.Create(argc, c_argv))

cdef object PyImageNetFactory(imageNet* ptr):
    cdef PyImageNet py_obj = PyImageNet()
    py_obj.c_net = ptr
    return py_obj

My setup.py:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

setup(
      cmdclass = {"build_ext": build_ext},
      ext_modules = [
          Extension("cy_wrapper",
                    sources=["cy_wrapper.pyx"],
                    libraries=["shared_inference"],
                    language="c++",
                    extra_compile_args=["-O3", "-Wall"],
                    extra_link_args=["-L../build/"]
                    )
          ]
)

Ofc libshared_inference.so is located in ../build/ and defines the class imageNet.

Upvotes: 3

Views: 1633

Answers (1)

Romzie
Romzie

Reputation: 457

For those interested, here is how I converted my list of string into a char** (I forgot to mention in my question that my method was static, but it does not really matter regarding my solution), hope this will help someone.

# cy_wrapper.pyx
from libc.stdlib cimport malloc, free

cdef extern:
    cdef cppclass imageNet:
        @staticmethod
        imageNet* Create(int argc, char** argv)

cdef class PyImageNet:
    cdef imageNet* c_net
    @staticmethod
    def Create(args):
        # Declare char**
        cdef char** c_argv
        # Allocate memory
        c_argv = <char**>malloc(len(args) * sizeof(char*))
        # Check if allocation went fine
        if c_argv is NULL:
            raise MemoryError()
        # Convert str to char* and store it into our char**
        for i in range(len(args)):
            args[i] = args[i].encode()
            c_argv[i] = args[i]
        # Grabbing return value
        cdef imageNet* c_tmp_net = imageNet.Create(len(args), c_argv)
        # Let him go
        free(c_argv)
        # Return python-compatible value
        return PyImageNetFactory(c_tmp_net)

cdef object PyImageNetFactory(imageNet* ptr):
    cdef PyImageNet py_obj = PyImageNet()
    py_obj.c_net = ptr
    return py_obj

Test code:

# test.py
import cy_wrapper
args = ["str1", "str2"]
net = cy_wrapper.PyImageNet.Create(args)

Upvotes: 1

Related Questions