CODEWITHSUNDEEP
pythonarraysnumpymultidimensional-arraycython
user3433489
user3433489

Reputation: 989

cython: pass a 2D numpy array to cdef function

I want to pass a 2D numpy array to a cdef function, where the dimensions of the array can vary. Here's what I tried:

cimport numpy as cnp

input = numpy.array([[3.34, 2.2],[1.1, -0.6]])
input = input[:,:].astype(np.double)
cdef int nrows = 2
cdef int ncols = 2

# output of function
cdef cnp.ndarray[cnp.uint8_t, ndim=2] output = np.zeros((2,2), dtype=np.uint8)  

test_array(nrows, ncols, &input[0], <unsigned char**>output.data)

where my test_array function is:

cdef void test_array(Py_ssize_t nrows, Py_ssize_t ncols, double **x, unsigned char **output) nogil:

output[0][0]=1
output[1][0]=0
output[1][1]=1
output[0][1]=0

and my function prototype is:

cdef void test_array(Py_ssize_t nrows, Py_ssize_t ncols, double **x, unsigned char **output) nogil

When I compile, I get an error saying " Cannot take address of Python object" and pointing to &input[0]. That syntax works for a 1D array, but I'm not sure what the syntax is for a 2d array. I also tried &input[0][0] but that's wrong too.

Upvotes: 5

Views: 5986

Answers (1)

ead
ead

Reputation: 34316

It is not clear what you would like to achieve:

A: if it should be a pure cython function then you should use typed memory view, that means your function signature should be

cdef void test_array(double[:,:] x, unsigned char[:,:] output) nogil:

There are no nrows, ncols because the typed memory views have this information (similar to std::vector).

B: array_test is actually a wrapper for a c-function, which expects double ** and unsigned char **, then you should take a look at this SO-question.

Actually, I would like to explain, why your attempts didn't work.

First, why didn't &input[0] work? The real question is what is input[0]:

import numpy as np
input=np.zeros((3,3))
type(input[0])
<type 'numpy.ndarray'>
type(input[:,0])
<type 'numpy.ndarray'>
type(input[0,0])
<type 'numpy.float64'>

so input is a numpy.ndarray that means a python object, and cython refuses to take its address. The same is the case for the input[0,0] - it is a python object. No luck so far.

To get it work, you need the input to be a cython-numpy array (I don't know how to express it better - take a look at the example):

import numpy as np
cimport numpy as np #that the way it is usually imported

def try_me():
    cdef np.ndarray[double, ndim=2] input = np.array([[3.34, 2.2],[1.1, -0.6]])
    cdef double *ptr1=&input[0,0]
    cdef double *ptr2=&input[1,0]
    print ptr1[0], ptr2[1] #prints 3.34 and -0.6

The important part: input is no longer considered/interpreted as a python object but as of type cython-type np.ndarray[double, ndim=2] and this is what makes the syntax &input[0,0] possible in the first place.

Maybe a more precise way to see it like: cimport numpy gives us additional tools in handling of numpy arrays so we can access internals which are not accessible in pure python.

However, &input[0,0] is not of type double ** but of type double *, because numpy.ndarray is just a continuous chunk of memory and only the operator [i,j] mocks the feeling of 2d:

How it feels:
    A[0] -> A00 A01 A02
    A[1] -> A10 A11 A12

The real layout in the memory:
     A00 A01 A02 A10 A11 A12

There are no pointers to rows, but you could create them via cdef double *ptr2=&input[row_id,0], how it could be handled is discussed in the above mentioned question.

To say that numpy.ndarray is just a continuous piece of memory is a simplification - numpy.ndarray is quite a complicated beast! Please consider the following example:

import numpy as np
cimport numpy as np

def try_me2():
    cdef np.ndarray[double, ndim=2] input = np.array([[1.0, 2.0],
                                                      [3.0, 4.0]])
    cdef np.ndarray[double, ndim=1] column = input[:,1]  
    cdef double *ptr = &column[0]
    print column          #prints column "2 4"
    print ptr[0],ptr[1]   #prints "2 3" and not "2 4"!

Now, here input and column share the same memory and in the memory input[1][0] is saved after input[0][1]=column[0] and only then input[1][1]=column[1]. ptr[1] takes the memory cell next to input[0][1] and this is input[1][0]=3 and not input[1][1]=4.

Upvotes: 7

Related Questions