CODEWITHSUNDEEP
pythonmultithreadinggarbage-collection
Paul R
Paul R

Reputation: 2797

Killing Thread and releasing memory in Python

I'm killing thread like shown here: Is there any way to kill a Thread in Python?

But I noticed, that memory is not released (gc.get_objects() keeps growing and growing). In fact these objects are lists, dicts etc., not files.

Id there any way to manually release resources? Code:

import ctypes

def terminate_thread(thread):
    """Terminates a python thread from another thread.

    :param thread: a threading.Thread instance
    """
    if not thread.isAlive():
        return

    exc = ctypes.py_object(SystemExit)
    res = ctypes.pythonapi.PyThreadState_SetAsyncExc(
        ctypes.c_long(thread.ident), exc)
    if res == 0:
        raise ValueError("nonexistent thread id")
    elif res > 1:
        # """if it returns a number greater than one, you're in trouble,
        # and you should call it again with exc=NULL to revert the effect"""
        ctypes.pythonapi.PyThreadState_SetAsyncExc(thread.ident, None)
        raise SystemError("PyThreadState_SetAsyncExc failed")

class MyThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        self.result = None
        self.error = None

    def run(self):
        try:
            self.result = myfun(*args, **kw) #run external resource and the interrupt it
        except Exception as e:
            self.error = e

c = MyThread()
c.start()
c.join(60) # wait a minute
counter = 0
if c.isAlive():
    while c.isAlive():
        time.sleep(0.1)
        try:
            terminate_thread(c) # how to release resources?
        except:
            break
        counter += 1
        if counter > 10: break
    raise TimeoutException

Example of output of:

print('Controlled objects: %s' % len(gc.get_objects()))
print ('Unreachable: %s' % gc.collect())

Controlled objects: 85084 #request 1, no timeout
Unreachable: 5640

Controlled objects: 171994 # request 2, timeout
Unreachable: 7221

Upvotes: 3

Views: 4514

Answers (1)

Joran Beasley
Joran Beasley

Reputation: 114038

ok after all this garbage I think what you want is the multiprocessing module as I believe you can actually send a sigkill on that

class MyThread:
    def __init__(self):
        self.result = None
        self.error = None
    def start(self):
        self.proc = multiprocessing.Process(target=self.run)
        self.proc.start()
    def stop(self):
       self.proc.send_signal(multiprocessing.SIG_KILL)
    def run(self):
        try:
            self.result = myfun(*args, **kw) #run external resource and the interrupt it
        except Exception as e:
            self.error = e

then you would call c.stop() in order to halt the thread with the sig_kill (of coarse the other thing should respond appropriately to this)

you can probably even just use the builtin subprocess.Process.kill() (see the docs https://docs.python.org/2/library/subprocess.html#subprocess.Popen.send_signal)

WRT YOUR QUESTION##

(Id there any way to manually release resources? )

 t = Thread(target=some_long_running_external_process)
 t.start()

there is no way to exit your thread (t) from outside of some_long_running_external_process

Upvotes: 4

Related Questions