CODEWITHSUNDEEP
pythoncgarbage-collectionsegmentation-faultrefcounting
Tommy
Tommy

Reputation: 620

c python extension causes python script to exit with segfault due to garbage collection error

I have the following routine in C that I use to turn an array of strings into a python list of strings

PyObject* build_pylist(char** strings, unsigned int string_cnt){

    PyObject* list = PyList_New(string_cnt);
    int i;
    for(i = 0; i < string_cnt; i++){

        PyObject* pystring = PyString_FromStringAndSize(
            (const char*) strings[i], 
            (Py_ssize_t) strlen(strings[i])
        );

        #per http://www.kbs.twi.tudelft.nl/Documentation/Programming/python-2.1/ext/thinIce.html
        #apparently the inc/dec is necessary...doesn't seem
        #to make a difference

        Py_INCREF(pystring);
        PyList_SET_ITEM(
            list, 
            (Py_ssize_t) i, 
            pystring
        );
        Py_DECREF(pystring);

        free(strings[i]);
    }

    free(strings);
    return list;
}

The PyString_FromStringAndSize function makes a copy of the given string, so I free the unnecessary strings as they are copied, and then free the container of those strings' pointers. This all seems to work fine. The python list is returned up to the script, the strings all look good, and the refcounts look good when checked via sys.getrefcount on the strings in the list, and the list itself.

The refcounts all return 2 which seems right because the call to getrefcount causes an increment of 1 temporarily. I'm pretty sure it's got something to do with the ref counting based on core dump analysis

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   org.python.python               0x000000010abfc52a collect + 482
1   org.python.python               0x000000010abfc33f PyGC_Collect + 35
2   org.python.python               0x000000010abea056 Py_Finalize + 290
3   org.python.python               0x000000010abfbe9b Py_Main + 3143
4   libdyld.dylib                   0x00007fff8843a7e1 start + 1

The error is occurring at the exit of the script and you can clearly see the crash is happening in the garbage collector. Bad refcounting is the only thing I can think of for this error.

Any thoughts?

Upvotes: 1

Views: 537

Answers (1)

Dennis Sakva
Dennis Sakva

Reputation: 1467

Not directly related to garbage collection, but are you using the same compiler that was used to build your python version? Sometimes it matters. I was once debugging segfault for several weeks which went away only when I replaced MinGW with Microsoft Visual C++ Compiler for python.

Upvotes: 0

Related Questions