CODEWITHSUNDEEP
pythonc++ccython
Carpetfizz
Carpetfizz

Reputation: 9169

Renamed .c files to .cpp, importing Cython library fails

I had a working Cython program which wrapped some C libraries and custom C code. Recently, I had to switch my project to C++, so I renamed all my C code to *.cpp. Cython compiled fine and produced the .so file. However, when I try to import my library in Python, I get the following error.

File "example.py", line 1, in <module>
    from tag36h11_detector import detect
ImportError: dlopen(/Users/ajay/Documents/Code/calibration/apriltag_detector/examples/tag36h11_detector.cpython-36m-darwin.so, 2): Symbol not found: _free_detection_payload
  Referenced from: /Users/ajay/Documents/Code/calibration/apriltag_detector/examples/tag36h11_detector.cpython-36m-darwin.so
  Expected in: flat namespace
 in /Users/ajay/Documents/Code/calibration/apriltag_detector/examples/tag36h11_detector.cpython-36m-darwin.so

Because I'm not sure about the source of the error, I'm not sure what relevant information to provide.

Here's my setup.py

from distutils.core import setup, Extension
from Cython.Build import cythonize
import numpy

setup(ext_modules=cythonize(Extension(
    name='tag36h11_detector',
    sources=["tag36h11_detector.pyx",
             "tag36h11_detector/tag36h11_detector.cpp"],
    include_dirs=["/usr/local/include/apriltag", numpy.get_include()],
    libraries=["apriltag"])))

I compile it with python setup.py build_ext --inplace

Thanks for any assistance!

Upvotes: 2

Views: 523

Answers (1)

ead
ead

Reputation: 34347

Add language=c++ to your setup:

setup(ext_modules=cythonize(Extension(
    name='XXX',
    ....
    language="c++",
 )))

You probably use gcc. The frontend of gcc (and many other compilers) decides whether the file is compiled as C (cc1 is used) or C++ (cc1plus is used) depending on its extension: ".c" is C, ".cpp" is C++.

If you use extra_compile_args=["-v"], in your setup you can see exactly which compiler is used:

  1. Cython creates "tag36h11_detector.c" and because of its extension the C-compiler (cc1) is used.
  2. For the file "tag36h11_detector/tag36h11_detector.cpp" the C++-compiler (cc1plus) is used.`

One of the differences between C and C++ is the name mangling: C expects that the names of the symbols in the object files are not mangled, but C++ mangles it.

For example for a function with signature int test(int) C tells to the linker to search for a symbol called test, but C++ creates a symbol called _Z4testi instead, and thus it cannot be find in the linkage step.

Now, what happens during the linkage? On Linux, the default-behavior of linking a shared object is, that we can have undefined symbols. It is implicitly assumed, that those symbols will be availably during the run-time, when the shared library is loaded. That means the program fails only when the shared object is loaded and the symbol cannot be found, i.e. when you import your module.

You could add extra_link_args=["-Wl,--no-undefined"] to ensure that the compilation fails if there are undefined symbols ain order to not have any surprises during the runtime.

One way to fix it could be too say to C++-compiler to emit unmangled names using extern "C" in your code, as pointed out in the comments.

A less intrusive approach would be to make clear to compiler, that C++-convention is used by adding language="c++" to the setup.

With language="c++", Cython creates "XXX.cpp" (instead of "XXX.c") from "XXX.pyx", and thus gcc chooses C++-compiler for the cythonized file, which is aware of the right name-mangling.

Upvotes: 3

Related Questions