Thread in python-

Question

I want to use threads to get better performance in python.

My program need to return value from each function the thread does.

And I need to know when the thread is finished.

There are 3 ways I tried to execute this little program.

import thread
import datetime
from threading import Thread
import threading
from multiprocessing.pool import ThreadPool


def func1(word):
    i=0
    while i<100000:
        if 1<2:
            i=i+1
    return "func1"


def func2():
    i=0
    while i<100000:
        if 1<2:
            i=i+1
    return "func2"

word="hiiii"

"""
#--------------------------------example1--------------------------------
pool = ThreadPool(processes=2)

print str(datetime.datetime.now().time())
async_result1 = pool.apply_async(func1, (word, ))

async_result2 = pool.apply_async(func2)

print async_result1.get()  
print  async_result2.get()  
print str(datetime.datetime.now().time())


print func1(word)
print func2()
print str(datetime.datetime.now().time())
#with threads-71000
#without threads- 40000
#--------------------------------example1--------------------------------
"""

"""
#--------------------------------example2--------------------------------

t1 = Thread(target=func1, args=(word,))
t2 = Thread(target=func2, args=())
print str(datetime.datetime.now().time())

t1.start()
t2.start()

t1.join()
t2.join()
print str(datetime.datetime.now().time())

func1(word)
func2()
print str(datetime.datetime.now().time())

#with threads-75000
#without threads-42000
#--------------------------------example2--------------------------------
"""

"""
#--------------------------------example3 without sending value--------------------------------
print str(datetime.datetime.now().time())

t1 = threading.Thread(name=func1,target=func1)
t2= threading.Thread(name=func2,target=func2)
t1.start()
t2.start()

t1.join()
t2.join()
print str(datetime.datetime.now().time())
func1()
func2()
print str(datetime.datetime.now().time())

#with threads- 73000
#without threads- 42000
#--------------------------------example3 without sending value------------- -------------------
   """

But, you can see that the better way to run is without threads! why?? what do I do wrong? How to use threads?

Answer 1

To append the other answer:

The reason the threaded version can actually be slower in a CPU-bound operation is (as mentioned) GIL, the Global Interpreter Lock. GIL works as a mutex, allowing only a single thread to access it at a time. So threading is only useful when waiting for IO-operations.

However, there is a way to actually execute code in multiple cores, bypassing the issue with GIL. The solution is to use multiprocessing library.

So in your case:

process1 = multiprocessing.Process(target=func1, args=(word,))
process1.start()
process2 = multiprocessing.Process(target=func2)
process2.start()

process.join()
process2.join()

The two processes should start in different cores, effectively making the program execute faster. For me it almost halved the execution time. Note that spawning more processes than you have cores will again make it run slower.

Answer 2

Threading is primarily of use when you have multiple tasks contending for CPU but spending most of their time waiting for some external event like a network read or a database write.

With multiple threads active, every (some number) of opcodes it spends time deciding to switch to another thread (assuming there are other runnable threads). If neither of your threads ever does anything but compute then there is effectively no time to be saved by switching between threads. Because all threads run in the same process, and a Python process cannot (normally) take advantage of multiple CPUs, you won't see any speed-up (and, indeed, may observe slow-down due to thread-switching activity).

tl;dr: CPU-bound tasks cannot be sped up by multi-threading.