CODEWITHSUNDEEP
pythonwindows-7numpycython
Noob Saibot
Noob Saibot

Reputation: 4749

Cython: "fatal error: numpy/arrayobject.h: No such file or directory"

I'm trying to speed up the answer here using Cython. I try to compile the code (after doing the cygwinccompiler.py hack explained here), but get a fatal error: numpy/arrayobject.h: No such file or directory...compilation terminated error. Can anyone tell me if it's a problem with my code, or some esoteric subtlety with Cython?

Below is my code. 

import numpy as np
import scipy as sp
cimport numpy as np
cimport cython

cdef inline np.ndarray[np.int, ndim=1] fbincount(np.ndarray[np.int_t, ndim=1] x):
    cdef int m = np.amax(x)+1
    cdef int n = x.size
    cdef unsigned int i
    cdef np.ndarray[np.int_t, ndim=1] c = np.zeros(m, dtype=np.int)

    for i in xrange(n):
        c[<unsigned int>x[i]] += 1

    return c

cdef packed struct Point:
    np.float64_t f0, f1

@cython.boundscheck(False)
def sparsemaker(np.ndarray[np.float_t, ndim=2] X not None,
                np.ndarray[np.float_t, ndim=2] Y not None,
                np.ndarray[np.float_t, ndim=2] Z not None):

    cdef np.ndarray[np.float64_t, ndim=1] counts, factor
    cdef np.ndarray[np.int_t, ndim=1] row, col, repeats
    cdef np.ndarray[Point] indices

    cdef int x_, y_

    _, row = np.unique(X, return_inverse=True); x_ = _.size
    _, col = np.unique(Y, return_inverse=True); y_ = _.size
    indices = np.rec.fromarrays([row,col])
    _, repeats = np.unique(indices, return_inverse=True)
    counts = 1. / fbincount(repeats)
    Z.flat *= counts.take(repeats)

    return sp.sparse.csr_matrix((Z.flat,(row,col)), shape=(x_, y_)).toarray()

Upvotes: 193

Views: 139500

Answers (9)

Mudasir Habib
Mudasir Habib

Reputation: 848

Simple

$ python
import numpy as np
np.get_include()

'/Users/mudasirhabib/Documents/projects/verticalsols/tts/venv/lib/python3.9/site-packages/numpy/core/include'*

Now in terminal enter

 export CFLAGS="-I/Users/mudasirhabib/Documents/projects/verticalsols/tts/venv/lib/python3.9/site-packages/numpy/core/include $CFLAGS"

Thats all

Upvotes: 3

Pedro Henrique Nogueira
Pedro Henrique Nogueira

Reputation: 11

I hadn't sudo privileges on the server I was running and export CFLAGS didn't work with me. For sake of simplicity, I've installed Anaconda ( https://docs.anaconda.com/anaconda/install/) which creates links to all its installed packages, including Numpy. You can also install miniconda and work with environments to avoid using too much space.

Upvotes: -1

taras
taras

Reputation: 6914

As per this answer, if you have installed numpy with pip on Linux, you will need to manually set a symbolic link to /usr/include/numpy

In my case the path is:

sudo ln -s /usr/local/lib/python3.8/dist-packages/numpy/core/include/numpy/ /usr/include/numpy

Upvotes: 5

Syrtis Major
Syrtis Major

Reputation: 3929

If you are too lazy to write setup files and figure out the path for include directories, try cyper. It can compile your Cython code and set include_dirs for Numpy automatically.

Load your code into a string, then simply run cymodule = cyper.inline(code_string), then your function is available as cymodule.sparsemaker instantaneously. Something like this

code = open(your_pyx_file).read()
cymodule = cyper.inline(code)

cymodule.sparsemaker(...)
# do what you want with your function

You can install cyper via pip install cyper.

Upvotes: 3

hsc
hsc

Reputation: 1298

It should be able to do it within cythonize() function as mentioned here, but it doesn't work beacuse there is a known issue

Upvotes: 1

strpeter
strpeter

Reputation: 2764

Simple answer

A way simpler way is to add the path to your file distutils.cfg. It's path behalf of Windows 7 is by default C:\Python27\Lib\distutils\. You just assert the following contents and it should work out:

[build_ext]
include_dirs= C:\Python27\Lib\site-packages\numpy\core\include

Entire config file

To give you an example how the config file could look like, my entire file reads:

[build]
compiler = mingw32

[build_ext]
include_dirs= C:\Python27\Lib\site-packages\numpy\core\include
compiler = mingw32

Upvotes: 1

Robert Kern
Robert Kern

Reputation: 13430

In your setup.py, the Extension should have the argument include_dirs=[numpy.get_include()].

Also, you are missing np.import_array() in your code.

--

Example setup.py:

from distutils.core import setup, Extension
from Cython.Build import cythonize
import numpy

setup(
    ext_modules=[
        Extension("my_module", ["my_module.c"],
                  include_dirs=[numpy.get_include()]),
    ],
)

# Or, if you use cythonize() to make the ext_modules list,
# include_dirs can be passed to setup()

setup(
    ext_modules=cythonize("my_module.pyx"),
    include_dirs=[numpy.get_include()]
)

Upvotes: 270

Steve Byrnes
Steve Byrnes

Reputation: 2270

For a one-file project like yours, another alternative is to use pyximport. You don't need to create a setup.py ... you don't need to even open a command line if you use IPython ... it's all very convenient. In your case, try running these commands in IPython or in a normal Python script:

import numpy
import pyximport
pyximport.install(setup_args={"script_args":["--compiler=mingw32"],
                              "include_dirs":numpy.get_include()},
                  reload_support=True)

import my_pyx_module

print my_pyx_module.some_function(...)
...

You may need to edit the compiler of course. This makes import and reload work the same for .pyx files as they work for .py files.

Source: http://wiki.cython.org/InstallingOnWindows

Upvotes: 54

John Brodie
John Brodie

Reputation: 6019

The error means that a numpy header file isn't being found during compilation.

Try doing export CFLAGS=-I/usr/lib/python2.7/site-packages/numpy/core/include/, and then compiling. This is a problem with a few different packages. There's a bug filed in ArchLinux for the same issue: https://bugs.archlinux.org/task/22326

Upvotes: 23

Related Questions