CODEWITHSUNDEEP
pythonmultithreadingconcurrency
MappaGnosis
MappaGnosis

Reputation: 1169

Trying to understand Python's Concurrent.Futures module

I’m trying to understand how to use the concurrent.futures module in Python 3.2.2, and have been playing with examples from the documentation. When I try to apply my understanding, my own examples fail. I hope somebody can help me get on track!

I want to be able to set a number of processes running concurrently but asynchronously. My processes don't return anything. To simulate this I have written a trivial example:

import concurrent.futures

fred = [1,2,3,4,5,6,7,8,9,10]

def f(x):
    print(x * x)

def main():
    with concurrent.futures.ProcessPoolExecutor() as executor:
        for num in fred:
            executor.submit(f, num)

if __name__ == "__main__":
    main()

This code runs (on a 4 core Windows XP box) and returns:

1 4 9 16 25

...but then hangs.

Clearly I’m doing something wrong. So, what is the correct way to have Python run processes in a process pool? I don’t want to use the executor.map approach, because I don’t have any return from my process.

Or…do I have to fake it, by having the process return True or False (or something)?

Thanks!

Upvotes: 5

Views: 3045

Answers (1)

user35288
user35288

Reputation:

I don't quite understand why you don't want to use executor.map...I tried it with a function that didn't return anything (with your f function, actually) and it worked fine...

now, when you run it with map, it won't actually print the values, but here is a version of your f function that does print the values:

import concurrent.futures

fred = [1,2,3,4,5,6,7,8,9,10]

def f(x):
    return x * x

with concurrent.futures.ProcessPoolExecutor() as executor:
    for num in executor.map(f, fred):
        print(num)

Upvotes: 2

Related Questions