CODEWITHSUNDEEP
pythonout-of-memory
noname
noname

Reputation: 363

How can I make Python wait when it cannot allocate memory?

Sometimes a Python program stops with an exception like the following, when there is not enough memory:

OSError: [Errno 12] Cannot allocate memory

Can I make it wait until memory is available again instead of dying unrecoverably? Or at least freeze until the user sends a SIGCONT or something to it?

It's not my program, so I prefer not to modify its source code, but I think it would still be cool if I can do that by modifying only the outmost calling part.

Thank you!

Upvotes: 0

Views: 1051

Answers (1)

John Zwinck
John Zwinck

Reputation: 249394

You can catch the OSError exception but this may not help your program to continue where it left off.

To do this well you need to interpose some code between Python and malloc. You can do that using LD_PRELOAD as per the details here: How can I limit memory acquired with `malloc()` without also limiting stack?

The idea is you implement a wrapper for malloc which calls the real malloc and waits to retry if it fails. If you prefer not to use LD_PRELOAD then building Python with your interposing code baked in is a possibility (but a bit more work).

The library you'll write for LD_PRELOAD will end up being usable with just about any program written in C or C++. You could even open-source it. :)

Upvotes: 2

Related Questions