CODEWITHSUNDEEP
pythonmultiprocessing
Guy Blanc
Guy Blanc

Reputation: 861

Multiprocessing in Python slower than without

I'm using python 3.3 on a computer with 2 cores but 4 threads. I am trying to learn to use multiprocessing to speed up code, but when using it my code slows down.

To start my learning, I have made a small program:

from multiprocessing import Process
import time

def f():
    s = 0
    for i in range(2*10**7):
        s += i
    return s

if __name__ == '__main__':
    t = time.time()
    p1 = Process(target = f)
    p2 = Process(target = f)
    p3 = Process(target = f)
    p4 = Process(target = f)
    p1.start()
    p2.start()
    p3.start()
    p4.start()
    p1.join()
    p2.join()
    p3.join()
    p4.join()
    print (time.time()-t)

t2 = time.time()
for a in range(4):
    f()
print(time.time()-t2)

Average of 3 runs, the first part with multiprocessing takes 17.15 sec, while the second part without multiprocessing takes 6.24 sec. Using the Windows Task Manager, I see that my computer is indeed using 100% CPU for the first part and only 25% for the second part, and also that I am not running out of memory.

Why is this program so much slower with multiprocessing?

Upvotes: 1

Views: 725

Answers (1)

mata
mata

Reputation: 69092

Windows has no fork(), so multiprocessing has to work around that by importing the __main__ module each time a new process is started.

This means when each of your subprocesses runs, it doesn't only run the target function, but also the part at the end of the file. Move it into the block and it should be way faster!

Upvotes: 2

Related Questions