CODEWITHSUNDEEP
pythonconcurrencytornadoconcurrent.futures
MRocklin
MRocklin

Reputation: 57319

Use concurrent.futures with tornado event loop?

How can occasionally use threads and futures within a Tornado application?

My server occasionally needs to run long-runnings tasks in a separate thread or process (the task releases the GIL.) I would like to do this with a concurrent.futures exectutor

from concurrent.futures import ThreadPoolExecutor
executor = ThreadPoolExecutor(10)
future = executor.submit(func, *args, **kwargs)

How can I integrate this future into the Tornado event loop? Ideally this integration plays well with tornado.gen coroutines. I want to yield from the future without blocking on it. What is the best way to accomplish this?

Ideally I would like to yield from a concurrent Future.

I'm looking for a function f that makes the following possible

@gen.coroutine
def my_coroutine(...)
    ...
    future = executor.submit(func, *args, **kwargs)
    result = yield f(future)

Upvotes: 3

Views: 958

Answers (1)

A. Jesse Jiryu Davis
A. Jesse Jiryu Davis

Reputation: 24027

You don't need a special f function at all, simply yield the Future that executor.submit returns:

from concurrent.futures import ThreadPoolExecutor
executor = ThreadPoolExecutor(10)

@gen.coroutine
def my_coroutine(...)
    ...
    future = executor.submit(func, *args, **kwargs)
    result = yield future

Upvotes: 5

Related Questions