CODEWITHSUNDEEP
pythonmultithreading
barspurk
barspurk

Reputation: 29

python multiprocessing apply_async seems to be running jobs in series

I have been debugging this very simple code that calls two functions which each wait for 1 second before finishing. I know that apply_async is suppose to run jobs in parallel from a pool of workers. So if there are 10 workers and 20 wait commands submitted the program should run for a total of 2 seconds since 10 of the 1 sec wait commands should run in 10 processes in parallel. My code with 20 wait commands keeps running around 20 seconds suggesting to me that my attempt to parallelize my code isn't working. It runs for 20 seconds no matter the number of workers in my pool Does anyone have a suggestion for what I might be doing wrong? Thanks!

from multiprocessing import Pool

def doubler(number):
    time.sleep(1)

def double_trouble(number):
    time.sleep(1)

if __name__ == '__main__':
    start_time = time.time()
    pool = Pool(processes=10)

    for i in range(10):
        pool.apply_async(double_trouble(i))
        pool.apply_async(doubler(i))

    pool.close()
    pool.join()
    print ("We took second: ", time.time()-start_time)

Upvotes: 0

Views: 676

Answers (1)

bav
bav

Reputation: 1623

pool.apply_async takes function args as a separate param:

pool.apply_async(double_trouble, (i,))
pool.apply_async(doubler, (i,))

Upvotes: 1

Related Questions