CODEWITHSUNDEEP
pythongarbage-collectionsegmentation-faultcythongarbage
Krastanov
Krastanov

Reputation: 6549

Garbage collection segfault in python code while `gc.garbage` is empty

The following code works fine and prints out an empty list (on python3.4):

import gc
# code interfacing with C and cython
print(gc.garbage, flush=True)

Appending this line to the very end makes it segfault:

gc.collect()

It seems like it is pure luck that the automatic collection does not happen on its own (it took me some time to get reproducible error). The collection is not ran at "end of program" either because I am running this interactively.

Some googling led me to believe that some of the C/Cython code is creating/destroying objects without telling python that those objects are already removed from memory. Then python tries to remove them and crashes.

How can I find what those objects are? I thought that they would be in gc.garbage before the crash inducing gc.collect()?

Or are my assumptions completely incorrect?

Prepending an gc.set_debug(gc.DEBUG_STATS | gc.DEBUG_LEAK) to this code leads to a python exception instead of a segfault.

---> print(gc.garbage, flush=True)
     gc.collect()
...
ReferenceError: weakly-referenced object no longer exists

Edit: changed/simplified after comments from one of the answers

Upvotes: 0

Views: 1904

Answers (1)

user2357112
user2357112

Reputation: 280456

You're probably misunderstanding both gc.garbage and gc.disable(). gc.disable() turns off the cyclic garbage collector, leaving only reference-counting memory management active. There isn't some separate "garbage detector" that keeps detecting cyclic trash even when the garbage collector is turned off; garbage detection is the garbage collector's job.

Also, gc.garbage is only populated with specific kinds of weird garbage the GC couldn't clear. Most garbage never ends up there.

As for how you'd go about debugging this, I'm not familiar with the tools you'd use. Generic tools like Valgrind would probably be useful, and GC flags like gc.DEBUG_LEAK might help.

Upvotes: 1

Related Questions