How to automatically restart python to fix memory leak?

Question

I have a program that I want to run in python on about 20 cores (for a research math question) but the code has a bit of a memory leak. It is ok running small batches of this code but I would really like to just let it run on these cores for say a couple weeks. Is there any way in Python to build in an automatic restart or something to avoid the long-term memory issues? Delete function does not help. Any help is appreciated!

Answer 1

You can have a script that just starts and kills other python processes, but that seems like a weak workaround for your memory leak.

Take a look at this SO answer for debugging python memory leaks

Answer 2

first, persistent your compute status and result, then write a detect function to detect the memory leak, stop the program, using supervisor or pm2 to restart it.