CODEWITHSUNDEEP
pythonmultithreadingmultiprocessing
Bruce
Bruce

Reputation: 41

How to use Pool.join() in Python multiprocessing module?

Here is the code:

def function(index):
    print('start process '+str(index))
    time.sleep(1)
    print('end process '+str(index))
    return str(index)

if __name__ == '__main__':
    pool = Pool(processes=3)   
    for i in range(4):
        res = pool.apply_async(function,args=(i,)) 
        print(res.get())
    pool.close()
    print('done')

and the output:

start process 0
end process 0
0
start process 1
end process 1
1
start process 2
end process 2
2
start process 3
end process 3
3
done

In my opinion, if the I don't use the pool.join(), the code should only print 'done' and that's it, because the function of pool.join() is 'Wait for the worker processes to exit', but now without pool.join(), it get the same result. I really don't understand.

Upvotes: 3

Views: 3352

Answers (2)

Christian
Christian

Reputation: 1411

In your code, the method get() has the same effect as join(). It also waits for the process to finish because you want to get the result of it.

If you remove it from your code, you will see the 'done' being printed first:

done
start process 0

Upvotes: 4

Alex Hall
Alex Hall

Reputation: 36023

res.get waits for the process to finish (how else would it get the return value?) which means that process 0 must finish before process 1 can start, and so on.

Remove res.get and you won't see the processes finish. Move res.get to a separate loop after the first one and you'll see they all start before any of them finish.

Also check out Pool.map.

Upvotes: 1

Related Questions