Reputation: 4152
If I have understood correctly how the concurrent.futures
module in Python 3 works, the following code:
import concurrent.futures
import threading
# Simple function returning a value
def test(i):
a = 'Hello World\n'
return a
def main():
output1 = list()
with concurrent.futures.ThreadPoolExecutor() as executor:
# psdd iterator to test function
for out1 in executor.map(test, range(0, 10)):
# append returned result
output1.append(out1)
# confirm output
print(output1)
print("Task Executed {}".format(threading.current_thread()))
if __name__ == '__main__':
main()
...performs the following functions:
test()
.However, what I really want is to process the loop in parallel in my main()
function as so:
import concurrent.futures
import threading
def main():
output1 = list()
with concurrent.futures.ThreadPoolExecutor() as executor:
# psdd iterator to test function
for out1 in executor.submit(range(0, 10)):
a = 'Hello World\n'
# append returned result
output1.append(a)
# confirm output
print(output1)
print("Task Executed {}".format(threading.current_thread()))
if __name__ == '__main__':
main()
...this however produces the following error:
Traceback (most recent call last):
File "G:\HTPC Scripts\WebGrab Plus\TESTTESTTEST2.py", line 221, in <module>
main()
File "G:\HTPC Scripts\WebGrab Plus\TESTTESTTEST2.py", line 209, in main
for out1 in executor.submit(range(0, 10)):
TypeError: 'Future' object is not iterable
What do I need to amend in my code?
Thanks
Upvotes: 2
Views: 14754
Reputation: 43573
It doesn't matter whether you use submit
or map
, you always have to use a callable (such as a function) as the first argument.
Python does allow nested functions (also take note of the way to use Futures
);
import concurrent.futures
def main():
def worker(arg):
return str(arg) + ' Hello World!'
with concurrent.futures.ThreadPoolExecutor() as e:
fut = [e.submit(worker, i) for i in range(10)]
for r in concurrent.futures.as_completed(fut):
print(r.result())
if __name__ == '__main__':
main()
The only way to define a callable in-place is with a lambda
expression, but those have significant limitations.
Upvotes: 6