CODEWITHSUNDEEP
pythonnumpycython
MiKL
MiKL

Reputation: 73

Cython/Numpy - cimport numpy as np fails

I am trying to use cython to code critical part of my code but I have trouble doing to import numpy :

Setup.py 

    from distutils.core import setup
    from distutils.extension import Extension
    from Cython.Distutils import build_ext
    import numpy as np

    e_m1 = Extension('OrderC', ['OrderC.pyx'])
    ext_mods = [e_m1]
    setup(name = 'OrderC/ListOrderC',  cmdclass = {'build_ext': build_ext}, 
          include_dirs = [np.get_include()], ext_modules = ext_mods)

OrderC.pyx :

    import numpy as np
    #cimport numpy as np  ## CRITICAL LINE (RUN WITHOUT IT)

    cdef class OrderC:
        cdef double _px

    def __init__(self, double px):
        self._px = px

    def printPx(self):
        print self._px

Main :

    import pyximport; 
    pyximport.install()

    from OrderC import OrderC
    oo = OrderC(9.95)
    oo.printPx()

When I comment the critical line, the compiling print warnings but runs, here are the warning :

    C:\Users\user\.pyxbld\temp.win32-2.7\Release\pyrex\OrderC.c: In function '__Pyx_RaiseArgtupleInvalid':
    C:\Users\user\.pyxbld\temp.win32-2.7\Release\pyrex\OrderC.c:1264:18: warning: unknown conversion type character 'z' in format [-Wformat=]
              (num_expected == 1) ? "" : "s", num_found);
              ^
    C:\Users\user\.pyxbld\temp.win32-2.7\Release\pyrex\OrderC.c:1264:18: warning: format '%s' expects argument of type 'char *', but argument 5 has type 'Py_ssize_t' [-Wformat=]
    C:\Users\user\.pyxbld\temp.win32-2.7\Release\pyrex\OrderC.c:1264:18: warning: unknown conversion type character 'z' in format [-Wformat=]
    C:\Users\user\.pyxbld\temp.win32-2.7\Release\pyrex\OrderC.c:1264:18: warning: too many arguments for format [-Wformat-extra-args]

But, when I decomment the critical line, it is not working at all:

Building the setup file give me :

    running build_ext
    cythoning OrderC.pyx to OrderC.c
    building 'OrderC' extension
    C:\MinGW\bin\gcc.exe -mdll -O -Wall -IC:\Python27_x86\lib\site-packages\numpy\core\include -IC:\Python27_x86\include -IC:\Python27_x86\PC -c OrderC.c -o build\temp.win32-2.7\Release\orderc.o
    In file included from C:\Python27_x86\lib\site-packages\numpy\core\include/numpy/ndarraytypes.h:1804:0,
             from C:\Python27_x86\lib\site-packages\numpy\core\include/numpy/ndarrayobject.h:17,
             from C:\Python27_x86\lib\site-packages\numpy\core\include/numpy/arrayobject.h:4,
             from OrderC.c:352:
    C:\Python27_x86\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h:12:9: note: #pragma message: C:\Python27_x86\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
     #pragma message(_WARN___LOC__"Using deprecated NumPy API, disable it by " \
     ^
    OrderC.c: In function '__Pyx_RaiseArgtupleInvalid':
    OrderC.c:3828:18: warning: unknown conversion type character 'z' in format [-Wformat=]
              (num_expected == 1) ? "" : "s", num_found);
              ^
    OrderC.c:3828:18: warning: format '%s' expects argument of type 'char *', but argument 5 has type 'Py_ssize_t' [-Wformat=]
    OrderC.c:3828:18: warning: unknown conversion type character 'z' in format [-Wformat=]
    OrderC.c:3828:18: warning: too many arguments for format [-Wformat-extra-args]
    OrderC.c: In function '__Pyx_RaiseTooManyValuesError':
    OrderC.c:4065:18: warning: unknown conversion type character 'z' in format [-Wformat=]
              "too many values to unpack (expected %" CYTHON_FORMAT_SSIZE_T "d)", expected);
              ^
    OrderC.c:4065:18: warning: too many arguments for format [-Wformat-extra-args]
    OrderC.c: In function '__Pyx_RaiseNeedMoreValuesError':
    OrderC.c:4071:18: warning: unknown conversion type character 'z' in format [-Wformat=]
              index, (index == 1) ? "" : "s");
              ^
    OrderC.c:4071:18: warning: format '%s' expects argument of type 'char *', but argument 3 has type 'Py_ssize_t' [-Wformat=]
    OrderC.c:4071:18: warning: too many arguments for format [-Wformat-extra-args]
    In file included from C:\Python27_x86\lib\site-packages\numpy\core\include/numpy/ndarrayobject.h:26:0,
             from C:\Python27_x86\lib\site-packages\numpy\core\include/numpy/arrayobject.h:4,
             from OrderC.c:352:
    OrderC.c: At top level:
    C:\Python27_x86\lib\site-packages\numpy\core\include/numpy/__multiarray_api.h:1629:1: warning: '_import_array' defined but not used [-Wunused-function]
     _import_array(void)
     ^
    In file included from C:\Python27_x86\lib\site-packages\numpy\core\include/numpy/ufuncobject.h:317:0,
             from OrderC.c:353:
    C:\Python27_x86\lib\site-packages\numpy\core\include/numpy/__ufunc_api.h:241:1:         warning: '_import_umath' defined but not used [-Wunused-function]
     _import_umath(void)
     ^
    writing build\temp.win32-2.7\Release\OrderC.def
    C:\MinGW\bin\gcc.exe -shared -s build\temp.win32-2.7\Release\orderc.o build\temp.win32-2.7\Release\OrderC.def -LC:\Python27_x86\libs -LC:\Python27_x86\PCbuild -lpython27 -lmsvcr90 -o C:\Users\user\Desktop\Workspace\TestC\OrderC.pyd

So no critical error but a lot of (weirds) warnings.

Execution :

When I execute the main, it gives me :

    C:\Users\mmahfouda\.pyxbld\temp.win32-2.7\Release\pyrex\OrderC.c:352:31: fatal error: numpy/arrayobject.h: No such file or directory
     #include "numpy/arrayobject.h"
                           ^
    compilation terminated.
    Traceback (most recent call last):
      File "C:\Users\mmahfouda\Desktop\Workspace\TestC\MainTestC.py", line 4, in <module>
from OrderC import OrderC
      File "C:\Python27_x86\lib\site-packages\pyximport\pyximport.py", line 431, in load_module
language_level=self.language_level)
      File "C:\Python27_x86\lib\site-packages\pyximport\pyximport.py", line 209, in load_module
inplace=build_inplace, language_level=language_level)
      File "C:\Python27_x86\lib\site-packages\pyximport\pyximport.py", line 186, in build_module
reload_support=pyxargs.reload_support)
      File "C:\Python27_x86\lib\site-packages\pyximport\pyxbuild.py", line 104, in pyx_to_dll
dist.run_commands()
      File "C:\Python27_x86\lib\distutils\dist.py", line 953, in run_commands
self.run_command(cmd)
      File "C:\Python27_x86\lib\distutils\dist.py", line 972, in run_command
cmd_obj.run()
      File "C:\Python27_x86\lib\site-packages\Cython\Distutils\build_ext.py", line 163, in run
_build_ext.build_ext.run(self)
      File "C:\Python27_x86\lib\distutils\command\build_ext.py", line 337, in run
self.build_extensions()
      File "C:\Python27_x86\lib\site-packages\Cython\Distutils\build_ext.py", line 171, in build_extensions
self.build_extension(ext)
      File "C:\Python27_x86\lib\distutils\command\build_ext.py", line 496, in build_extension
depends=ext.depends)
      File "C:\Python27_x86\lib\distutils\ccompiler.py", line 574, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
      File "C:\Python27_x86\lib\distutils\cygwinccompiler.py", line 166, in _compile
raise CompileError, msg
    ImportError: Building module OrderC failed: ["CompileError: command 'C:\\\\MinGW\\\\bin\\\\gcc.exe' failed with exit status 1\n"]

For information, my framework is : * Windows 7 * Pydev * Python 2.7 (32 bits)

Do you have the same error or have an idea why this simple program doesn't work ?

Thank you.

Upvotes: 3

Views: 2336

Answers (1)

MiKL
MiKL

Reputation: 73

I post the solution suggested byt ever which worked for me :

As he said, the problem comes from compiling with pyximport when you use extra C libraries as numpy.

I have modified the main file as follows :

from OrderC import OrderC
oo = OrderC(9.95)
oo.printPx()

Then,

  1. Compile with Cython :

    C:\Python27_x86\Scripts\cython.exe OrderC.pyx

  2. Compile the c file with gcc (including extra libraries)

    C:\MinGW\bin\gcc.exe -mdll -O -Wall -IC:\Python27_x86\lib\site-packages\numpy\core\include -IC:\Python27_x86\include -IC:\Python27_x86\PC -c OrderC.c -o OrderC.o

  3. Compile the c file with gcc (including extra libraries)

    C:\MinGW\bin\gcc.exe -shared -s OrderC.o -LC:\Python27_x86\libs -LC:\Python27_x86\PCbuild -lpython27 -lmsvcr90 -o C:\Users\user\Desktop\Workspace\TestC\OrderC.pyd

Then it works.

NB :

  • step 2 generates warnings but works

  • make sure to add OrderC path in your PATH environment variable

Upvotes: 2

Related Questions