CODEWITHSUNDEEP
pythonmultithreadingpython-3.x
user2109254
user2109254

Reputation: 1759

Python multiprocessing starts processes, but only one is active

I got the following code from the accepted answer in the following question. When I run it up on my windows 10 x64bit machine, in Python 3.5.1 it starts the threads but only one of them shows any CPU usage, and it is very little at that. While this is happening the terminal is scrolling up numbers from the print statement so it is running... When I run other python scripts that use loops I see 13.5% CPU usage for the script. I expected to see this spin up 4 threads each with 13.5% CPU usage. I am running the script via windows command prompt and and these are the only 5 python processes running. When I exist the script by pressing Ctrl+C all the processes stop.

Why is this not working as expected?

enter image description here

Question: How do I "multi-process" the itertools product module?

Code From Accepted Answer:

#!/usr/bin/env python3.5
import sys, itertools, multiprocessing, functools


alphabet = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ12234567890!@#$%^&*?,()-=+[]/;"
num_parts = 4
part_size = len(alphabet) // num_parts

def do_job(first_bits):
    for x in itertools.product(first_bits, alphabet, alphabet, alphabet):
        print(x)

if __name__ == '__main__':
    pool = multiprocessing.Pool(processes=4)
    results = []
    for i in range(num_parts):
        if i == num_parts - 1:
            first_bit = alphabet[part_size * i :]
        else:
            first_bit = alphabet[part_size * i : part_size * (i+1)]
        results.append(pool.apply_async(do_job(first_bit)))

    pool.close()
    pool.join()

Thanks for your time.

Upvotes: 1

Views: 2603

Answers (1)

Daniel Roseman
Daniel Roseman

Reputation: 599946

You're not running anything asynchronously here; you call the functions before passing to the async workers. You should pass the callable, not the result:

pool.apply_async(do_job, (first_bit,))

Also note that the return value from apply_async is an AsyncResult instance: when you do come to calculating things in the workers, you'll need to call get() on those values - again, after all the processes have finished - to actually get the results.

Upvotes: 4

Related Questions