CODEWITHSUNDEEP
pythonmultithreadinggarbage-collectiondestructor
datenwolf
datenwolf

Reputation: 162297

Python threading.Thread, scopes and garbage collection

Say I derive from threading.Thread:

from threading import Thread

class Worker(Thread):
    def start(self):
        self.running = True
        Thread.start(self)

    def terminate(self):
        self.running = False
        self.join()

    def run(self):
        import time
        while self.running:
            print "running"
            time.sleep(1)

Any instance of this class with the thread being started must have it's thread actively terminated before it can get garbage collected (the thread holds a reference itself). So this is a problem, because it completely defies the purpose of garbage collection. In that case having some object encapsulating a thread, and with the last instance of the object going out of scope the destructor gets called for thread termination and cleanup. Thuss a destructor

    def __del__(self):
        self.terminate()

will not do the trick.

The only way I see to nicely encapsulate threads is by using low level thread builtin module and weakref weak references. Or I may be missing something fundamental. So is there a nicer way than tangling things up in weakref spaghetti code?

Upvotes: 6

Views: 7972

Answers (3)

AndreasT
AndreasT

Reputation: 9801

I guess you are a convert from C++ where a lot of meaning can be attached to scopes of variables, equalling lifetimes of variables. This is not the case for Python, and garbage collected languages in general. Scope != Lifetime simply because garbage collection occurs whenever the interpreter gets around to it, not on scope boundaries. Especially as you are trying to do asynchronuous stuff with it, the raised hairs on your neck should vibrate to the clamour of all the warning bells in your head! You can do stuff with the lifetime of objects, using 'del'. (In fact, if you read the sources to the cpython garbage collector module, the obvious (and somewhat funny) disdain for objects with finalizers (del methods) expressed there, should tell everybody to use even the lifetime of an object only if necessary).

You could use sys.getrefcount(self) to find out when to leave the loop in your thread. But I can hardly recommend that (just try out what numbers it returns. You won't be happy. To see who holds what just check gc.get_referrers(self)). The reference count may/will depend on garbage collection as well.

Besides, tying the runtime of a thread of execution to scopes/lifetimes of objects is an error 99% of the time. Not even Boost does it. It goes out of its RAII way to define something called a 'detached' thread. http://www.boost.org/doc/libs/1_55_0/doc/html/thread/thread_management.html

Upvotes: 1

Henry Gomersall
Henry Gomersall

Reputation: 8712

To add an answer inspired by @datenwolf's comment, here is another way to do it that deals with the object being deleted or the parent thread ending:

import threading
import time
import weakref

class Foo(object):

    def __init__(self):
        self.main_thread = threading.current_thread()

        self.initialised = threading.Event()
        self.t = threading.Thread(target=Foo.threaded_func,
                args=(weakref.proxy(self), ))

        self.t.start()
        while not self.initialised.is_set():
            # This loop is necessary to stop the main threading doing anything
            # until the exception handler in threaded_func can deal with the 
            # object being deleted.
            pass

    def __del__(self):
        print 'self:', self, self.main_thread.is_alive()
        self.t.join()

    def threaded_func(self):

        self.initialised.set()

        try:
            while True:
                print time.time()

                if not self.main_thread.is_alive():
                    print('Main thread ended')
                    break

                time.sleep(1)

        except ReferenceError:
            print('Foo object deleted')

foo = Foo()
del foo
foo = Foo()

Upvotes: 3

jwd
jwd

Reputation: 11144

How about using a wrapper class (which has-a Thread rather than is-a Thread)?

eg:

class WorkerWrapper:
    __init__(self):
        self.worker = Worker()
    __del__(self):
        self.worker.terminate()

And then use these wrapper classes in client code, rather than threads directly.

Or perhaps I miss something (:

Upvotes: 5

Related Questions