CODEWITHSUNDEEP
pythonpipcython
Nate
Nate

Reputation: 1948

Compile Cython on pip package build

I'm developing a Python package, EcoPy, that is mostly written in pure Python. The main folder is called ecopy. There's a subfolder called regression that has a Cython file that's already been built. The main setup.py file includes the code:

ext_modules = cythonize([
    Extension(
        'ecopy.regression.isoFunc', ['ecopy/regression/isoFunc.pyx'], **opts),
])

When I run

sudo pip install ecopy -e . --upgrade --force-reinstall

the module builds fine. It even re-compiles the isoFunc.c file if I've deleted it. The problem is that Cython doesn't then convert the .c file to the .so file, which I need in order to import the function. If I try loading the module without it, I get

ImportError: No module named isoFunc

If I manually setup file using the command line

python setup.py build_ext --inplace

Cython DOES generate the .so file. How do I get it to generate the .so file using pip? I've tried to figure out how statsmodels did it by reading their code, but honestly, its a mystery to me.

It's almost as if the pip command misses the build_ext argument.

Upvotes: 11

Views: 3946

Answers (2)

bluenote10
bluenote10

Reputation: 26550

I came here because I wasn't sure if pip installing my package is sufficient to automatically compile the Cython sources. It looks like it does but by default the pip output doesn't seem to log that. I noticed that pip install -e . -v makes it very clear if / what exactly is done regarding the compile and link steps:

Running setup.py develop for mypackage
  [...]
  running build_ext
  cythoning mypackage/my_cython_module.pyx to mypackage/my_cython_module.c
  building 'mypackage.my_cython_module' extension
  x86_64-linux-gnu-gcc [...] -c mypackage/my_cython_module.c -o build/temp.linux-x86_64-2.7/mypackage/my_cython_module.o
  x86_64-linux-gnu-gcc [...] build/temp.linux-x86_64-2.7/mypackage/my_cython_module.o -o build/lib.linux-x86_64-2.7/mypackage/my_cython_module.so
  copying build/lib.linux-x86_64-2.7/mypackage/my_cython_module.so -> mypackage
  [...]

Upvotes: 2

Nate
Nate

Reputation: 1948

I can answer this question, because I just learned that I'm an idiot.

sudo pip install ecopy -e . --upgrade --force-reinstall

was using an older version from the PyPI that didn't have the new setup.py with the Cython code. When I did it correctly

sudo pip install -e . --upgrade --force-reinstall

and used the latest version on my hard drive, it worked fine.

Little victories.

Upvotes: 7

Related Questions