CODEWITHSUNDEEP
linuxcython
deasmhumnha
deasmhumnha

Reputation: 456

Cython: Calling an extension type cdef method from a cdef function called by python

I'm trying to write a Cython module that calculates pairwise distances as part of a larger class of locality sensitive hashes. Instead of writing code for each type and each distance metric, I am attempting to create one cdef function that takes various extension types that inherit from Metric:

cdef class Metric:
    def __init__(self):
        pass

cdef class Euclidean(Metric):
    cdef numeric c_evaluate(self, numeric[:] x, numeric[:] y, int dims):
        ....

cdef numeric[:,:] pairwise(numeric[:] x, numeric[:] y, Metric func, bint symmetric):
    ...
    dm[i,j] = func.c_evaluate(x,y,dims)
    ...

To access this function from Python:

def py_pairwise(numeric[:,:] x, numeric[:,:] y, str func, bint symmetric = 1, **kwargs):
    cdef Metric mfunc = to_Metric(func, **kwargs)
    return pairwise(x, y, mfunc, symmetric)

However I keep getting the error that "c_distance.[Metric] object has no attribute 'c_evaluate'". I'm wondering if the c_evaluate method isn't accessible because the class object is created in python code through the python function to_Metric, though I thought def and cdef functions were supposed to be able to call each other freely within a Cython module. The method works if I change c_evaluate to a cpdef method, but I'm not sure if this fixes the problem by allowing the cdef object to pass through python to cython or simply uses the slower python method. Any suggestions (I'm also not at my home computer so I don't have all the code right now. Will update later/on request)?

Edit: That typo isn't in the original functions (there could still be others):

ctypedef fused floating:
    float
    double

cdef class Euclidean(Metric):
    cdef public floating c_evaluate(self, floating[:] x, floating[:] y, int dims):
        cdef int i
        cdef floating tmp, d = 0
        for i in range(dims):
            tmp = x[i]-y[i]
            d += tmp*tmp
        return sqrt(d)

#@cython.boundscheck(False)
#@cython.wraparound(False)
def py_pairwise(numeric[:,::1] x, numeric[:,::1] y,str metric, bint symmetric,**kwargs):
    cdef Metric func = to_Metric(metric,**kwargs)   
    return pairwise(x,y,func,symmetric)

cdef numeric[:,::1] pairwise(numeric[:,::1] x,numeric[:,::1] y, Metric met, bint symmetric):#
    cdef int n,m,k,i,j
    n = x.shape[0]
    m = y.shape[0]
    dims = x.shape[1]
    if numeric in floating:
        mdtype = np.float
    else:
        mdtype = np.int
    #mdtype = np.float
    cdef numeric[:,::1] dm = (np.empty((n,m),dtype = mdtype)).fill(0)
    if symmetric:
        interval = lambda i,n,m: range(i+1,m)
    else:
        interval = lambda i,n,m: range(m)
    for i in range(n):
        for j in interval(i,n,m):
            dm[i,j] = met.c_evaluate(x[i,:],y[j,:],dims)
    return np.asarray(dm)

Also, to_Metric: 

def to_Metric(str m, **kwargs):
    if len(kwargs) == 0:
        if m == 'euclidean':
            met = Euclidean()
        elif m in {'cos','cosine'}:
            met = Cosine()
        elif m in {'hamming','matching'}:
            met = Hamming()
        else:
            raise ValueError('Unrecognized metric {}'.format('\''+m+'\''))
    else:
        if m in {'pnorm','p-norm'}:
            met = Pnorm(kwargs['p'])
        elif m == 'maximal':
            met = Maximal(kwargs['m1'],kwargs['m2'],kwargs['sep'])
        else:
            raise ValueError('Unrecognized metric {}'.format('\''+m+'\''))
    return met

Upvotes: 0

Views: 963

Answers (1)

DavidW
DavidW

Reputation: 30889

The issue is that c_evaluate is associated with the class Euclidean and because of this can only be used with objects that are known to be of type Euclidean. However, in pairwise you declare the type of met to be Metric.

Because you declared c_evaluate function as cdef it can only be found at compile time. If you want the c_evaluate to be found at runtime like a standard Python function, you should declare it as def.

If you need the function to be found at compile time (which makes calling it quicker) then you should either make c_evaluate be a function of the Metric object, or you should make pairwise only take a Euclidean object.

Upvotes: 1

Related Questions