Reputation: 57319
How can occasionally use threads and futures within a Tornado application?
My server occasionally needs to run long-runnings tasks in a separate thread or process (the task releases the GIL.) I would like to do this with a concurrent.futures exectutor
from concurrent.futures import ThreadPoolExecutor
executor = ThreadPoolExecutor(10)
future = executor.submit(func, *args, **kwargs)
How can I integrate this future into the Tornado event loop? Ideally this integration plays well with tornado.gen
coroutines. I want to yield from the future without blocking on it. What is the best way to accomplish this?
Ideally I would like to yield from a concurrent
Future
.
I'm looking for a function f
that makes the following possible
@gen.coroutine
def my_coroutine(...)
...
future = executor.submit(func, *args, **kwargs)
result = yield f(future)
Upvotes: 3
Views: 958
Reputation: 24027
You don't need a special f
function at all, simply yield the Future that executor.submit
returns:
from concurrent.futures import ThreadPoolExecutor
executor = ThreadPoolExecutor(10)
@gen.coroutine
def my_coroutine(...)
...
future = executor.submit(func, *args, **kwargs)
result = yield future
Upvotes: 5