Tracking a nasty memory leak in python3 C extension

Question

I'm the author of libgpiod and in the most recent release I provided a set of object-oriented python3 bindings implemented as a C extension module.

The complete code for the module can be found here.

Recently a user reported a memory leak in the module. Since then I've been trying to debug it and managed to find and fix some other memory related problems but not the culprit of this exact leak.

Below is the script that the reporter used to trigger the problem:

#!/usr/bin/env python3

import gpiod
import logging
import os
import psutil
import sys
import time

this_process = psutil.Process(os.getpid())
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

chip = gpiod.Chip('1', gpiod.Chip.OPEN_BY_NUMBER)
gpio_line = chip.get_line(12)
gpio_line.request(consumer="test", type=gpiod.LINE_REQ_DIR_OUT)
count = 0
mem_used_prev = 0

while True:
    mem_used = this_process.memory_info().rss
    if mem_used != mem_used_prev:
        logging.info('count: {}  memory usage: {}'.format(count, this_process.memory_info().rss))
        mem_used_prev = mem_used

    gpio_line.set_value(1)
    count += 1

Example output:

2018-07-19 11:21:13,505 - INFO - count: 0  memory usage: 13459456
2018-07-19 11:21:13,516 - INFO - count: 638  memory usage: 14008320
2018-07-19 11:21:13,529 - INFO - count: 1298  memory usage: 14278656
2018-07-19 11:21:13,543 - INFO - count: 1958  memory usage: 14548992
2018-07-19 11:21:13,557 - INFO - count: 2618  memory usage: 14819328
2018-07-19 11:21:13,569 - INFO - count: 3278  memory usage: 15089664
2018-07-19 11:21:13,583 - INFO - count: 3938  memory usage: 15360000
2018-07-19 11:21:13,596 - INFO - count: 4598  memory usage: 15630336
2018-07-19 11:21:13,611 - INFO - count: 5258  memory usage: 15900672

Every couple iterations there's a sudden surge in memory usage. I guess this is when the heap is resized, but the actual leak probably happens every iteration.

While investigating I noticed that the leak happens for all operations using a single GPIO line which involves packaging this single object into a LineBulk object representing a set of GPIO lines - this is done to reuse the code so that gpiod_Line_set_value() can simply call gpiod_LineBulk_set_values() for a set consisting of a single line.

Next I noticed that the leak also happens when calling chip.get_lines() which also needs to create a LineBulk object.

With that I believe that the leak happens somewhere in gpiod_LineBulk_init() which is implemented as follows:

static int gpiod_LineBulk_init(gpiod_LineBulkObject *self, PyObject *args)
{
    PyObject *lines, *iter, *next;
    Py_ssize_t i;
    int rv;

    rv = PyArg_ParseTuple(args, "O", &lines);
    if (!rv)
        return -1;

    self->num_lines = PyObject_Size(lines);
    if (self->num_lines < 1) {
        PyErr_SetString(PyExc_TypeError,
                "Argument must be a non-empty sequence");
        return -1;
    }
    if (self->num_lines > GPIOD_LINE_BULK_MAX_LINES) {
        PyErr_SetString(PyExc_TypeError,
                "Too many objects in the sequence");
        return -1;
    }

    self->lines = PyMem_RawCalloc(self->num_lines, sizeof(PyObject *));
    if (!self->lines) {
        PyErr_SetString(PyExc_MemoryError, "Out of memory");
        return -1;
    }

    iter = PyObject_GetIter(lines);
    if (!iter) {
        PyMem_RawFree(self->lines);
        return -1;
    }

    for (i = 0;;) {
        next = PyIter_Next(iter);
        if (!next) {
            Py_DECREF(iter);
            break;
        }

        if (next->ob_type != &gpiod_LineType) {
            PyErr_SetString(PyExc_TypeError,
                    "Argument must be a sequence of GPIO lines");
            Py_DECREF(next);
            Py_DECREF(iter);
            goto errout;
        }

        self->lines[i++] = next;
    }

    self->iter_idx = -1;

    return 0;

errout:

    if (i > 0) {
        for (--i; i >= 0; i--)
            Py_DECREF(self->lines[i]);
    }
    PyMem_RawFree(self->lines);
    self->lines = NULL;

    return -1;
}

This function takes a sequence of Line objects and packages it in a LineBulk object which implements a set of methods to allowing to manipulate the GPIOs.

I've been trying to figure out the culprit using various tools. Tracemalloc didn't help much as it doesn't go into C code. I tracked PyObject_Malloc's and Free's and calls to relevant destructors with gdb but everything seems to be ok, objects seem to be destroyed when needed. Valgrind doesn't report any leaks either.

I'm currently out of ideas and don't have much experience with python C API either. Any suggestions are greatly appreciated.

Answer 1

Just to close this question: I figured out the problem. It was because of not calling PyObject_Del() as the last action of the destructor.