Wrapping a function returning a Vector of Complex types in Cython

Question

I'm currently trying to wrap a C++ class I have created. One of the functions returns a vector of vectors which contain complex floats:

std::vector<std::vector<std::complex<float>>>

My DataBridge.pxd file is shown below:

# distutils: language = c++

from libcpp cimport bool
from libcpp cimport float
from libcpp.vector cimport vector
from libcpp.complex cimport complex

from libc.float cimport float

cimport numpy as np
import numpy as np

cdef extern from "projectxcpp/bridge.cpp":
    pass

# Declare the class with cdef
cdef extern from "projectxcpp/bridge.hpp":
    cdef cppclass DataBridge:
        DataBridge() except +
        void start()
        void stop()
        bool isDataReady()
        vector[vector[complex[float]]] getData()

My bridge.hpp file is shown below:

#ifndef BRIDGE_HPP
#define BRIDGE_HPP

#include <vector>
#include <complex>

...

class DataBridge {
    public:
        int start(void);
        int stop(void);

        bool isDataReady(void);
        std::vector<std::vector<std::complex<float>>> getData(void);

    private:
        ...
};

#endif

Then, trying to run the setup.py, the following message is given:

[1/1] Cythonizing snowconecpp.pyx

Error compiling Cython file:
------------------------------------------------------------
...
        DataBridge() except +
        void start()
        void stop()
        bool isDataReady()
        vector[vector[complex[float]]] getData()
                            ^
------------------------------------------------------------

DataBridge.pxd:24:29: Array size must be a compile time constant

Error compiling Cython file:
------------------------------------------------------------
...
        DataBridge() except +
        void start()
        void stop()
        bool isDataReady()
        vector[vector[complex[float]]] getData()
                            ^
------------------------------------------------------------

DataBridge.pxd:24:29: unknown type in template argument
Traceback (most recent call last):
  File "setup.py", line 15, in <module>
    setup(ext_modules=cythonize("projectxcpp.pyx", language_level="3"))
  File ".../lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1096, in cythonize
    cythonize_one(*args)
  File ".../lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1219, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: projectxcpp.pyx

My setup.py file contains simply:

from distutils.core import setup

from Cython.Build import cythonize

setup(ext_modules=cythonize("projectxcpp.pyx", language_level="3"))

When declaring types in my wrapper as either complex[float] e or vector[vector[float]] e, these will parse and move to the next step of compilation.

Just wondering if there's a special requirement for putting a complex type within vectors for Cython - cannot seem to find answers for this or similar situations where C++ wrapping is involved.

My end goal is to be able to read this data out into a numpy array. If there is a better way of going about this, please make me aware of it too - Thank you!

Answer 1

I think the issue is that complex is already a type in Python/Cython (and one that's slightly badly handled in places too, due to the overlap with C or C++ complex...). It works if you either rename it, or address it by it's full path

from libcpp.complex cimport complex as cpp_complex

# later
vector[vector[cpp_complex[float]]]

or

cimport libcpp.complex

# later
vector[vector[libcpp.complex.complex[float]]]

---

For the more general part of the question: my preference would be not to provide the data for a Numpy array like that. Part of this is that I really dislike 2D arrays being defined as pointer-to-pointer or vector-of-vector. You're better off using a 1D array/vector stored alongside shape information that lets you index it.

There's a few options for interfacing with Numpy:

1. If you know the size in advance then allocate the memory with Numpy and pass that memory into C++ (you can get a pointer to the first element of a Numpy array). This way the C++ code modifies Numpy's memory and you don't have to copy the data.

2. If you want to allocate the memory with C++ then you can wrap it with a Cython class that implements the buffer protocol. The memory is managed by C++, but Numpy can directly access the memory without copying. (You may need to implement a move constructor to avoid copying, but that is fairly straightforward).

3. You can use Numpy C API functions PyArray_SimpleNewFromData to get Numpy to wrap the memory. This requires care to ensure that it's freed at the right time.

4. You can do what you're doing currently and copy the data over. This is very simple to implement, but does add a copy operation every time you want to move data from C++ to Python.