CODEWITHSUNDEEP
pythonnumpymultiprocessinganaconda
Tian Qin
Tian Qin

Reputation: 11

python multiprocessing pool run slower?

I'm learning the python multiprocessing for numeric calculation, here is the sample code I implemented, I guess that the problem should be the wrong usage of map arguments.

def pow(a):
    c = np.zeros(a.size)
    for i in range(a.size):
        c[i] = a[i] ** a[i]
    return c        

if __name__=='__main__':
    n = int(1e6)

    data = np.random.sample(n)
    pool = mp.Pool(processes=8)
    results = pool.map(pow, data)
    new_res = pow(data)

Upvotes: 1

Views: 75

Answers (1)

oekopez
oekopez

Reputation: 1090

Parallel computing comes never for free. The new processes need to be created and the results need to be sent around the processes. Python loops are slow in comparison with loops written in C, as you can use them in numpy. Parallel computing is only helping for large problems that cannot be optimized in another way. Your example is so simple, working in parallel will not speed your problem up in comparison to:

a = np.random.sample(n)
c = a ** a

Upvotes: 1

Related Questions