Reputation: 3299
I have a question on best practice on how to use code from a local C library with Cython. In order to avoid setting the LD_LIBRARY_PATH
(or installing the C library in /usr/lib or similar directories), I think static linking would be a suitable solution for my use case.
However, interfacing the c functions that have been statically linked is non-trivial. The only way I could cimport
another cython module that has statically linked C code in it was to paraphrase all exported functions with Cython function pointers (see https://github.com/HolgerPeters/cython-example/blob/master/cython-project/c_with_ptrs.pyx and https://github.com/HolgerPeters/cython-example/blob/master/cython-project/c_with_ptrs.pxd)
Just declaring the original C function in the pxd as extern does not seem to work. I have exemplary project on Github which isolates different approaches and their usage.
https://github.com/HolgerPeters/cython-example
Any ideas on this? Can I tell Cython to explicitly use the extern
ed declarations?
As the question started as a rather open one, I would like to make it more specific to encourage answers.
In the named github project I have assembled several ways of linking. I will focus now on the way that seems natural from the documentation.
cdef extern from "foo.h": extern int clib_return_3(int)
and a use case in the corresponding https://github.com/HolgerPeters/cython-example/blob/master/cython-project/c_from_with_direct_compilation.pyx
def useit(): print(clib_return_3(4))
Invocation of c_from_with_direct_compilation.useit() works (test.sh output below).
But then when I try to use clib_return_3
from another cython file, it fails! Usage is
cimport c_from def useit(): print(c_from.clib_return_3(4))
bash test.sh
tests all usage cases, and as you can see, the cython so where the clib_return_3 symbol is linked into can indeed use that symbol, whereas the other cython so trying to import that symbol fails at this.
======================== c_from_w/_direct_comp, interfacing fails (why?) Invoke from statically linked c_from_with_direct_compilation, invocation by c_from_with_direct_compilation_user Traceback (most recent call last): File "", line 1, in ImportError: ./c_from_with_direct_compilation_user.so: undefined symbol: clib_return_3 FAILURE ======================== c_from_w/_direct_comp Invoke from statically linked c_from_with_direct_compilation 3 SUCCESS
Upvotes: 0
Views: 896
Reputation: 3299
After staring long enough at this problem I think I know what is going on:
Cython-cdef-functions are name mangled, whereas extern functions are not name-mangled and keep their original names. Name mangling is necessary for discoverability of functions across different modules. So non-mangled names cannot be shared via cimport
- for those, the normal C shared library mechanisms apply.
Upvotes: 0
Reputation: 3139
You can do either of:
cdef extern from "path/header.h":
and then list the function defs you'll use in cython. (http://docs.cython.org/src/userguide/external_C_code.html#referencing-c-header-files)extern
. (http://docs.cython.org/src/userguide/external_C_code.html#external-declarations)(Both methods work in pyx
and pxd
files).
Side note: in your setup.py
, you're using 2 different ways to include the C code (using the dynamic library & using the object file). Unless your C make rules are really complicated, a third option is to just list the .c
files in the same array as the .pyx
files and let distutils build them for you.
Upvotes: 1