CODEWITHSUNDEEP
pythoncythonmemoryview
Simeon Carstens
Simeon Carstens

Reputation: 3

How to avoid PyObject_to_MemoryviewSlice, GOTREF / DECREF Python API calls?

I have a problem with cythonizing my code, more specifically the following (and similar) sniplets:

cdef double [:,:] grad_d_him_d_jm
grad_d_ihm_d_jm = grad_d_im_d_jm(...)

where grad_d_im_d_jm(...) would return a double [:,:] memoryview. This code would be translated by Cython into the following C code:

__pyx_t_1 = __pyx_f_24gradient_better_c_mviews_grad_d_im_d_jm(__pyx_v_i, __pyx_v_j, __pyx_v_m, __pyx_v_structure, __pyx_v_distances); 
if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 203;  __pyx_clineno = __LINE__; goto __pyx_L1_error;}
__Pyx_GOTREF(__pyx_t_1);
__pyx_t_7 = __Pyx_PyObject_to_MemoryviewSlice_dsds_double(__pyx_t_1);
if (unlikely(!__pyx_t_7.memview)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 203; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
__Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
__pyx_v_grad_d_ihm_d_jm = __pyx_t_7;
__pyx_t_7.memview = NULL;
__pyx_t_7.data = NULL;

As I'm doing this in loops, I suspect the Python API calls to have quite some impact on the speed of my code.

the GOTREF / DECREF call occurs also on other occasions, along with a PyFloat_asFloat:

cdef float sp
sp = scalar_product()

where scalar_product() returns a cdef float. This snippet gets translated to

__pyx_t_1 = __pyx_f_24gradient_better_c_mviews_scalar_product(__pyx_v_i, __pyx_v_j, __pyx_v_m, __pyx_v_structure); 
if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 178; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
__Pyx_GOTREF(__pyx_t_1);
__pyx_t_2 = __pyx_PyFloat_AsFloat(__pyx_t_1); 
if (unlikely((__pyx_t_2 == (float)-1) && PyErr_Occurred())) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 178; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
__Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
__pyx_v_sp = __pyx_t_2;

I am running Python 2.7.11+ and Cython 0.23.4. I would be very grateful if you could either tell me that a) this is not relevant for performance or b) how to fix it. Let me know if I can improve the question, I'd be happy to do so.

Upvotes: 0

Views: 693

Answers (1)

Sergei Lebedev
Sergei Lebedev

Reputation: 2679

These appear to be part of Cython reference counting API explained here.

My guess is that grad_d_im_d_jm returns a Python object (e.g. NumPy array) and thus Cython has to decrement the object reference counter after the memoryview was obtained.

As for scalar_product, I think it is either a def (rather than cdef) or is untyped. For example the following 

cdef g():
    return 1.0

compiles to

// ...
__Pyx_XDECREF(__pyx_r);
__Pyx_INCREF(__pyx_float_1_0);
__pyx_r = __pyx_float_1_0;
goto __pyx_L0;

However, once you specify the return type the refcounting calls are gone

cdef float g():
    return 1.0

becomes

// ...
__pyx_r = 1.0;
goto __pyx_L0;

Upvotes: 2

Related Questions