CODEWITHSUNDEEP
pythoncython
conjectures
conjectures

Reputation: 841

Cython memoryview error - Fatal Python error: Acquisition count is

I'm encountering an error with Cython that I find difficult to resolve. I have a struct ret_val which has a long[:] field called last_visited. I'm trying to set this but get the following runtime error:

Fatal Python error: Acquisition count is -1753032536 (line 5052)

The following is an extract from the responsible C file at the line above:

/* "cymain.pyx":196
*     last = np.array([1,1,1], dtype=np.int64)
*     ret_val.last_visited = last             # <<<<<<<<<<<<<<
*/

__pyx_t_9 = __Pyx_PyObject_to_MemoryviewSlice_ds_long(__pyx_v_last);

if (unlikely(!__pyx_t_9.memview)) {
    __pyx_filename = __pyx_f[0]; 
    __pyx_lineno = 196; 
    __pyx_clineno = __LINE__; 
    goto __pyx_L1_error;
}
__PYX_XDEC_MEMVIEW(&__pyx_v_ret_val->last_visited, 0);
__pyx_v_ret_val->last_visited = __pyx_t_9;
__pyx_t_9.memview = NULL;
__pyx_t_9.data = NULL;

I tried making a minimal example to reproduce the error, but then it didn't happen. Then I rewrote the function this is drawn from using the minimal example, and it failed again.

Here's a minimal example which doesn't produce the error but is, as far as I can understand, functionally identical to the error causing code:

cdef struct baz:
    long[:] lv
    othermodule.something* cd

cdef baz* initialise_baz(dict req):
    cdef:
        baz* ret_val = <baz *> malloc(sizeof(baz))
        long nlevels = 3

    ret_val.cd = NULL

    lv = req["key"]
    lv = np.array(lv, dtype=np.int64)
    ret_val.lv = lv

    return ret_val

def test_memview_error(req):
    cdef baz* foo
    foo = initialise_baz(req)
    print "foo.lv[2]", foo.lv[2]

Then call

import cymodule
cymodule.test_memview_error({"key":np.array([1,2,3])})

Upvotes: 0

Views: 368

Answers (1)

DavidW
DavidW

Reputation: 30889

I believe the problem relates to uninitialised memory (as I said in the comments). Looking at the generated C code from your simple example:

/* "code.pyx":5
 * import numpy as np
 * 
 * cdef struct baz:             # <<<<<<<<<<<<<<
 *     long[:] lv
 *     #othermodule.something* cd
 */
struct __pyx_t_4code_baz {
  __Pyx_memviewslice lv;
};

(note that I've commented out othermodule.something for simplicity). __Pyx_memviewslice is defined as

typedef struct {
  struct __pyx_memoryview_obj *memview;
  char *data;
  Py_ssize_t shape[8];
  Py_ssize_t strides[8];
  Py_ssize_t suboffsets[8];
} __Pyx_memviewslice;

Some relevant code from initialise_baz (I've skipped some little bits here)

__pyx_v_ret_val = ((struct __pyx_t_4code_baz *)malloc((sizeof(struct __pyx_t_4code_baz))));

Note that malloc does not (necessarily) zero the memory. Thus the contents of lv (crucially the pointer to memview) are set to something arbitrary (likely what was in memory before - this obviously depends on what other code you've run before). If you use calloc instead of malloc it does zero the memory. initialise_baz continues:

  /* "code.pyx":18
 *     lv = req["key"]
 *     lv = np.array(lv, dtype=np.int64)
 *     ret_val.lv = lv             # <<<<<<<<<<<<<<
 * 
 *     return ret_val
 */
  __pyx_t_6 = __Pyx_PyObject_to_MemoryviewSlice_ds_long(__pyx_v_lv);
  if (unlikely(!__pyx_t_6.memview)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 18; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __PYX_XDEC_MEMVIEW(&__pyx_v_ret_val->lv, 0);
  __pyx_v_ret_val->lv = __pyx_t_6;
  __pyx_t_6.memview = NULL;
  __pyx_t_6.data = NULL;

The key line is __PYX_XDEC_MEMVIEW is called on the previous (arbitrary!) contents of lv. That's where things go wrong. lv.memview points to an arbitrary place, from which it reads what it thinks is an acquisition_count.

A superficial fix is the use calloc instead of malloc. However, even in that case lv never gets properly deallocated when you free baz, which probably causes a memory leak. I really don't think it makes sense to use memoryviews as part of C structures. Can you use them as part of a cdef class instead, where everything is taken care of properly?

Upvotes: 1

Related Questions