CODEWITHSUNDEEP
pythonc++cython
Lefix
Lefix

Reputation: 564

Python Cython Symbols are missing

I compiled the C++ rnnlib as shared library and want to call it from python. I chose cython and created a a c++ function void libCall(int argc, char* argv[]) (which is actually the same as the main function of rnnlib but renamed it to make it easier callable). The rnnlib library is inside /usr/lib

rnn.pyx

# distutils: language = c++

cdef extern from "libcall.hpp":
    void libCall(int argc, char* argv[])

cpdef call():
    print 'hallo welt'

setuprnn.py

from distutils.core import setup
from Cython.Build import cythonize
from distutils.extension import Extension
import os

os.environ["CC"] = "gcc"
os.environ["CXX"] = "g++"
os.environ["CFLAGS"]="-I./src/"

setup(ext_modules = cythonize(
       "rnn.pyx",
       libraries=["rnnlib","netcdf_c++","netcdf","m","stdc++"],
       language="c++",
      ))

Additional test.cpp

#include <iostream>
#include "libcall.hpp"

using namespace std;


int main(int argc, char* argv[]) {
    libCall(argc, argv);
}

Building test.cpp with

g++ -Wall -I./src/ test.cpp -lrnnlib -lnetcdf_c++ -lnetcdf -lm -lstdc++ -o test

works.

When I run python setuprnn.py build_ext -i I get rnn.so and rnn.cpp. But when trying to import rnn in Python:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: ./rnn.so: undefined symbol: _ZNK5NcVar3getEPcPKl

Inspecting rnn.so with nm yields:

000000000003f140 W _ZNK5Mdrnn5printERSo
             U _ZNK5NcDim4sizeEv
             U _ZNK5NcVar3getEPcPKl

This is strange, because this is the missing symbol.

Upvotes: 2

Views: 1691

Answers (2)

Lefix
Lefix

Reputation: 564

The problem are the dependencies of my libraries. Cython does not handle them.

The python script from above (setuprnn.py) does nothing else but execute this:

cython ...
gcc ...
g++ ... -lrnnlib -lnetcdf -lstdc++ -o rnn.so

This creates a problem as stated above in the entry post. The solution is then just to create a shell script for creating the library:

#!/bin/sh
cython ...
gcc ...
g++ ... -o rnn.so -lnetcdf_c++ -lnetcdf -lm -lstdc++ -lrnnlib

Upvotes: 0

hughdbrown
hughdbrown

Reputation: 49013

I can't replicate your problem. Here is the script I created to create the files, build the extension, and run the code in python:

#!/bin/sh -e

echo "Cleaning up /tmp"
rm -f /tmp/rnn.pyx /tmp/rnn.so /tmp/setuprnn.py /tmp/libcall.hpp /tmp/rnn.cpp /tmp/rnn.so
rm -rf /tmp/build/

echo "Creating /tmp/rnn.pyx"
cat > /tmp/rnn.pyx << EOF
# distutils: language = c++

cdef extern from "libcall.hpp":
    void libCall(int argc, char* argv[])

cpdef call():
    print 'hallo welt'
EOF

echo "Creating /tmp/setuprnn.py"
cat > /tmp/setuprnn.py << EOF
from distutils.core import setup
from Cython.Build import cythonize
from distutils.extension import Extension
import os

os.environ["CC"] = "gcc"
os.environ["CXX"] = "g++"
os.environ["CFLAGS"]="-I./src/"

setup(ext_modules = cythonize(
       "rnn.pyx",            # our Cython source
       libraries=["rnnlib","netcdf_c++","netcdf","m","stdc++"],  # additional source file(s)
       language="c++",             # generate C++ code
      ))
EOF

echo "Creating /tmp/libcall.hpp"
touch /tmp/libcall.hpp

echo "Building rnn.so extension"
python setuprnn.py build_ext -i

python -c "import rnn; print('-' * 30); print('Imported rnn'); print(dir(rnn))"

Here is the result I get:

Cleaning up /tmp
Creating /tmp/rnn.pyx
Creating /tmp/setuprnn.py
Creating /tmp/libcall.hpp
Building rnn.so extension
Compiling rnn.pyx because it changed.
Cythonizing rnn.pyx
running build_ext
building 'rnn' extension
creating build
creating build/temp.linux-x86_64-2.7
gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -I./src/ -fPIC -I/usr/include/python2.7 -c rnn.cpp -o build/temp.linux-x86_64-2.7/rnn.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -I./src/ build/temp.linux-x86_64-2.7/rnn.o -o /tmp/rnn.so
------------------------------
Imported rnn
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__test__', 'call']

Looks good to me. Of course, I don't have a correct implementation of all your code and nothing I wrote pulls in netcdf or the other libraries you cite. Maybe you could update your description so that others can test this out?

Upvotes: 2

Related Questions