Using Qt threads in pyside

Question

I don't understand the output of the sample code below found here. Current thread and worker thread have the same address, how is that possible?

from PySide import QtCore

class Master(QtCore.QObject):
    command = QtCore.Signal(str)
    def __init__(self):
        QtCore.QObject.__init__(self)

class Worker(QtCore.QObject):
    def __init__(self):
        QtCore.QObject.__init__(self)
    def do_something(self, text):
        print('in thread {} message {}'.format(QtCore.QThread.currentThread(), text))

if __name__ == '__main__':
    app = QtCore.QCoreApplication([])
    print(QtCore.QThread.currentThread())
    # give us a thread and start it
    thread = QtCore.QThread()
    thread.start()
    print(thread)

    # create a worker and move it to our extra thread
    worker = Worker()
    worker.moveToThread(thread)

    # create a master object and connect it to the worker
    master = Master()
    master.command.connect(worker.do_something)

    # call a method of the worker directly (will be executed in the actual thread)
    worker.do_something('in main thread')

    # communicate via signals, will execute the method now in the extra thread
    master.command.emit('in worker thread')

    # start the application and kill it after 1 second
    QtCore.QTimer.singleShot(1000, app.quit)
    app.exec_()

    # don't forget to terminate the extra thread
    thread.quit()

Output:

<PySide.QtCore.QThread object at 0x0000000002537688>
<PySide.QtCore.QThread object at 0x0000000002537688>
in thread <PySide.QtCore.QThread object at 0x00000000025377C8> message in main thread
in thread <PySide.QtCore.QThread object at 0x0000000002537688> message in worker thread

Answer 1

There's no "wrapper getting reused", just that the old wrapper object (that was deleted) happens to reside at the same memory address with the new wrapper object.

PySide uses shiboken library to wrap the QObjects into Python objects. The shiboken documentation says the following:

As any python binding, |project|-based bindings uses reference counting to handle the life of the wrapper object (the Python object that contains the C++ object, do not confuse with the wrapped C++ object). When a reference count reaches zero, the wrapper is deleted by Python garbage collector and tries to delete the wrapped instance, but sometimes the wrapped C++ object is already deleted, or maybe the C++ object should not be freed after the Python wrapper go out of scope and die, because C++ is already taking care of the wrapped instance.

Add to this the fact that the CPython malloc implementation (and many other common mallocs) often reuse the memory address that belonged to just deleted object, to subsequently created objects, leading to the common pitfall:

>>> {} is {}
False
>>> id({}) == id({})
True

Thus it is always so that the addresses/id's of live objects are distinct; the addresses of once lived and then dead objects are just a curiosity.

In the first case the reference for both dictionaries were held, so technically their id was distinct, whereas in the second case the left hand side dictionary is deleted soon after the left hand id is called, and only afterwards, the right hand dictionary is created, and allocated at the very same memory address.

---

However, it is a bit more complex yet. When you call the currentThread() it returns a new wrapper object for that QThread except if the old wrapper is still alive (meaning that it has references to it on the python side), in which case the old wrapper is returned: thus,

QtCore.QThread.currentThread() is QtCore.QThread.currentThread() 

will always be True!

---

Now, the reason why you get 2 addresses in

in thread <PySide.QtCore.QThread object at 0x00000000028EB888> message in main thread
in thread <PySide.QtCore.QThread object at 0x00000000028EB8C8> message in worker thread

again does not necessarily have nothing to do with the wrappers "not being reused", or them being held; instead the same kind of effect could happen with some other object(s) created in between, whose reference is being held - that is, one should not consider it certain that the wrapped object is being kept alive.

However, if you hold references to 2 distinct QThread objects, their id()s will be distinct for their lifetime.

---

All in all, you shouldn't rely on the addresses of the shiboken wrapper objects doing anything useful; instead if you yourself hold the reference in Python code, then the is test is useful

---

Note that __str__ of QObjects will print their objectName if it is set; thus you can make sense of the output easily by doing:

app = QtCore.QCoreApplication([])
QtCore.QThread.currentThread().setObjectName('main thread')

thread = QtCore.QThread()
thread.start()
thread.setObjectName('worker thread')

and with a clearer print:

print('in thread: "{}" - message: "{}"'
    .format(QtCore.QThread.currentThread().objectName(), text))

the messages will be:

in thread: "main thread" - message: "in main thread"
in thread: "worker thread" - message: "in worker thread"

---

There is also a way to fetch the address of the underlying C++ object, using the shiboken module; which would be handy for debugging other stuff too; alas, the Python library is not installed at all on Ubuntu 14.10 by default, or any of the shiboken-related packages; and after numerous answers I finally managed to install it by hand.

And the results are more scary than I imagined:

app = QtCore.QCoreApplication([])
print(QtCore.QCoreApplication.instance().thread())
obj = QtCore.QObject()
print(obj.thread())
del obj
print(QtCore.QThread.currentThread())

prints:

<PySide.QtCore.QThread object at 0x7fb4a1149cc8>
<PySide.QtCore.QThread object at 0x7fb4a1149cc8>
<PySide.QtCore.QThread object at 0x7fb4a1149d08>

That's right, after I delete the QObject the thread wrapper's address changes! So let's import dump from Shiboken.shiboken:

from Shiboken.shiboken import dump

def info(obj):
    print(id(obj))
    print(dump(obj))

app = QtCore.QCoreApplication([])
info(QtCore.QCoreApplication.instance().thread())

obj = QtCore.QObject()
info(obj.thread())    
del obj

info(QtCore.QCoreApplication.instance().thread())

The output is

140323585370568
C++ address....... PySide.QtCore.QThread/0xe3d880 
hasOwnership...... 0
containsCppWrapper 0
validCppObject.... 1
wasCreatedByPython 0
parent............ <PySide.QtCore.QCoreApplication object at 0x7f9fa175a948>

140323585370568
C++ address....... PySide.QtCore.QThread/0xe3d880 
hasOwnership...... 0
containsCppWrapper 0
validCppObject.... 1
wasCreatedByPython 0
parent............ <PySide.QtCore.QObject object at 0x7f9fa175aa48>

140323585370696
C++ address....... PySide.QtCore.QThread/0xe3d880 
hasOwnership...... 0
containsCppWrapper 0
validCppObject.... 1
wasCreatedByPython 0

That is, the shiboken makes the last object whose thread() method we called the parent of our thread. And whenever that object is deleted (a.k.a the object on which the .thread() was called last), the wrapper is deleted as well. Very dubious behaviour indeed; I am not sure if it is a bug, but this proves that trusting the ids of wrapper objects is not to be trusted at all.

Answer 2

I think it's not valid to use QThread.currentThread() in this context and expect a meaningful result.

As I understand the problem, QThread is not a thread, but a wrapper around a thread and what we see is just the wrapper that is getting reused for the worker thread. If I replace former QThread.currentThread() by QCoreApplication.instance().thread() I get the correct output:

<PySide.QtCore.QThread object at 0x00000000029D98C8>
<PySide.QtCore.QThread object at 0x00000000029D9908>
in thread <PySide.QtCore.QThread object at 0x00000000029D98C8> message in main thread
in thread <PySide.QtCore.QThread object at 0x00000000029D9908> message in worker thread

The reason behind this is very simple, the former call doesn't keep a reference on the QThread wrapper, unlike QCoreApplication.instance().thread() which does keep a reference. We can verify this by modifying original sample with:

main_thread = QtCore.QThread.currentThread() # holds the reference
print(main_thread)
# give us a thread and start it
thread = QtCore.QThread()
print(thread)

Outputs now:

<PySide.QtCore.QThread object at 0x00000000028EB888>
<PySide.QtCore.QThread object at 0x00000000028EB8C8>
in thread <PySide.QtCore.QThread object at 0x00000000028EB888> message in main thread
in thread <PySide.QtCore.QThread object at 0x00000000028EB8C8> message in worker thread

EDIT:

By wrapper reused, I was meaning that the memory block storing the first wrapper was reused by the second wrapper, that's why the wrapper has the same memory address, indeed.

But whay I'm interested in, is the mechanism behind this. And the reason is because of Python use PyMalloc for small objects (enabled by default since Python 2.3) but could also perfectly be a side effect of the glibc allocator itself (in case of modern allocators like jemalloc or tcmalloc and python compiled with flag --without-pymalloc):

http://www.evanjones.ca/memoryallocator/