CODEWITHSUNDEEP
pythonmultithreadingpython-multiprocessing
TommyT
TommyT

Reputation: 1797

Is threading module really parallel in python or do we need to use multiprocessing?

So, I have written this simple code to check if python threading module is really parallel and I found out that in this case,

from threading import Thread, current_thread
import multiprocessing as mp
def callback(result):
    print result

def run_sql(n):
    print current_thread()
    for i in range(n):
        i=i+1
    print 'done for ', current_thread()

if __name__=='__main__':
    n=100000000
    pool = mp.Pool(5)
    threads= [ Thread(target=run_sql, args=(n,)) for i in range(5)]
    for t in threads:
        t.start()
    for t in threads:
        t.join()

I have tried the same with Pool.apply_async which is actually parallel.

def callback(result):
    print result

def run_sql(n):
    print current_thread()
    for i in range(n):
        i=i+1
    print 'done for ', current_thread()

if __name__=='__main__':
    print datetime.datetime.now()
    n=100000000
    pool = mp.Pool(5)
    for i in range(10):
        pool.apply_async(run_sql, args= (n,),callback=callback)
    pool.close()
    pool.join()

So my question is what is the point of using the Threading module if it is not really parallel even it saves the IPC and uses the same memory area? Also, can threading actually be parallel using queue or something?

Upvotes: 2

Views: 486

Answers (1)

mgilson
mgilson

Reputation: 310059

It seems like you are joining your threads as soon as you start them:

for t in threads:
    t.start()
    t.join()

IIRC, Thread.join will wait for the thread to finish, before continuing (which means that you wait for the first thread to complete before starting the second)...

Generally, you need 2 loops:

for t in threads:
    t.start()
# other stuff to do in the main thread...
for t in thread:
    t.join()

Upvotes: 3

Related Questions