CODEWITHSUNDEEP
c++pythonmemory-leakspython-c-apireference-counting
linello
linello

Reputation: 8704

Python C Api and correct handling of memory in C++ classes

I'm wondering how to cope with the following problem. Inside my C++ class I have an auxiliary PyObject pointer.

class Foo
{
     public:
     // Should I new the dictionary here in constructor?
     Foo()
     {

     }
     // Must decrease the reference count or explicitly delete the dictionary?
     ~Foo()
     {
         Py_DECREF(myDictionary);
     }

     void sync()
     {
          myDictionary = PyDict_New();
          for (int i=0; i<myInternalData.size(); i++)
          {
                  PyObject *key =  PyInt_FromLong(i);
                  PyObject *val = PyInt_FromLong(myInternalData.at(i));
                  PyDict_SetItem(dict,key,val);
                  Py_DecRef(key);
                  Py_DecRef(val);
          }
     }

     private:
     PyObject *myDictionary;
     std::vector<int> myInternalData;
}

In my C++ code the myInternalData structure is occasionally updated or resized and I want to know how to cope with the proper memory allocation of my python dictionary.

I don't know how to deallocate the memory associated with it, or how to correctly keep it synchronized with my internal std::vector without corrupting the heap or provoking memory leaks.

Some helps with Python C API? Should I deallocate the PyDict with PyObject_Del and then reallocating it again? Someone suggesting another approach?

Upvotes: 0

Views: 1044

Answers (1)

James Kanze
James Kanze

Reputation: 153909

It's not clear to my why you're using a dictionary in Python, when you index with a contiguous set of integers, starting at 0. However: before using it, you'll have to do a PyDict_New to create the dictionary. Following that, when you resync, you should clear the dictionary before starting, using PyDict_Clear, rather than reallocate a new one.. Nothing else should be necessary. (If you reallocate a new one, as you do in your code, you should decrement the reference count on the old one first. But any code on the Python side which refers to the old one will continue to refer to the old one; PyDict_Clear is probably the better solution.)

Also, you should pay attention where temporary Python objects are involved. For the moment, nothing else is necessary because you only use Python (and thus, C) functions in the loop, and they cannot trigger a C++ exception. Change the code ever so slightly, and this may cease to be the case. As a general rule, I've found that you should wrap the PyObject* in a class whose destructor calls Py_DecRef, rather than call it explicitly, and maybe miss the call due to an exception.

Upvotes: 1

Related Questions