CODEWITHSUNDEEP
pythoncythonsetuptools
John Chrysostom
John Chrysostom

Reputation: 3963

Cython extension not re-installing via setup.py after modification

So, I have a Python package, with a Cython extension.

File structure:

-lib_name (dir)
    -setup.py
    -lib_name (dir)
        -__init__.py
        -python_thing.py
        -cython_thing.pyx

Here's my setup.py:

from setuptools import setup
from setuptools.extension import Extension
from Cython.Build import cythonize

extension = Extension(
    name="cython_thing",
    sources=["lib_name/cython_thing.pyx"],
    extra_compile_args=["-ffast-math"]
)

setup(
    ext_modules=cythonize(extension, language_level=3),
    name='lib_name',
    version='0.0.1',
    packages=['lib_name'],
    install_requires=['numpy', 'sklearn']
)

And here's my __init__.py:

from . import *

__all__ = ['python_thing', 'cython_thing']

I ran sudo python3 setup.py install one time, and everything seemed to go without a hitch. I saw the pyx file get cythonized and compiled, the package installed, and then I could seemingly use the whole package with no problem.

However, I've now made a modification to the .pyx file for some troubleshooting. When I re-install the package with install or develop, and attempt to use it, it continues to use the old version of the code. To confirm this, I've gone as far as adding print('THIS IS THE NEW CODE') to functions in the .pyx file to make sure it's identifying itself. However, these lines never print, telling me that I'm definitely still running the old code somehow. I do see Cython re-cythonizing the new code, but it must not be installing it, because it never runs.

I've already tried sudo pip3 uninstall lib_name, then re-installing, but no dice. Still get old code path.

In case it helps, I'm importing stuff from the .pyx file by using from lib_name.cython_thing import cython_function.

I'm fairly certain this must be a problem with my package structure or maybe some built object getting cached somewhere. Can anybody spot what I'm doing wrong? How do I ensure my package gets fully re-installed when needed?

Upvotes: 2

Views: 978

Answers (1)

ead
ead

Reputation: 34326

When developing, I usually have a virtual environment in which I install my cython package in developer mode via (in the folder with setup.py):

 pip install -e .

rerunning pip install -e . will update to the current state.

When using pip install (or setup.py install) one needs to uninstall the package, before it can be reinstalled.

The above is fast and thus convenient, however one has also to check installation in a clean virtual environment from time to time.

Upvotes: 1

Related Questions