CODEWITHSUNDEEP
pythonmultiprocessing
litd
litd

Reputation: 281

multiprocessing.Pool example

I'm trying to learn how to use multiprocessing, and found the following example.

I want to sum values as follows:

from multiprocessing import Pool
from time import time

N = 10
K = 50
w = 0

def CostlyFunction(z):
    r = 0
    for k in xrange(1, K+2):
        r += z ** (1 / k**1.5)
    print r
    w += r
    return r

currtime = time()

po = Pool()

for i in xrange(N):
    po.apply_async(CostlyFunction,(i,))
po.close()
po.join()

print w
print '2: parallel: time elapsed:', time() - currtime

I can't get the sum of all r values.

Upvotes: 28

Views: 83361

Answers (2)

GuySoft
GuySoft

Reputation: 1723

Here is the simplest example I found in the python example documentation:

from multiprocessing import Pool

def  f(x):
    return x*x

if __name__ == '__main__':
    pool = Pool(processes=4)              # start 4 worker processes
    result = pool.apply_async(f, [10])    # evaluate "f(10)" asynchronously
    print result.get(timeout=1)           # prints "100" unless your computer is *very* slow
    print pool.map(f, range(10))          # prints "[0, 1, 4,..., 81]"

It was simple enough even I could understand it.
Note result.get() is what triggers the computation.

Upvotes: 13

Justin Peel
Justin Peel

Reputation: 47072

If you're going to use apply_async like that, then you have to use some sort of shared memory. Also, you need to put the part that starts the multiprocessing so that it is only done when called by the initial script, not the pooled processes. Here's a way to do it with map.

from multiprocessing import Pool
from time import time

K = 50
def CostlyFunction((z,)):
    r = 0
    for k in xrange(1, K+2):
        r += z ** (1 / k**1.5)
    return r

if __name__ == "__main__":
    currtime = time()
    N = 10
    po = Pool()
    res = po.map_async(CostlyFunction,((i,) for i in xrange(N)))
    w = sum(res.get())
    print w
    print '2: parallel: time elapsed:', time() - currtime

Upvotes: 22

Related Questions