CODEWITHSUNDEEP
pythonpython-3.xasync-awaitfuturepython-asyncio
Tamas Hegedus
Tamas Hegedus

Reputation: 29916

Await Future from Executor: Future can't be used in 'await' expression

I wanted to use a ThreadPoolExecutor from a python coroutine, to delegate some blocking network calls to a separate thread. However, running the following code:

from concurrent.futures import ThreadPoolExecutor
import asyncio

def work():
  # do some blocking io
  pass

async def main():
  executor = ThreadPoolExecutor()
  await executor.submit(work)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()

causes error:

TypeError: object Future can't be used in 'await' expression

Aren't Future objects awaitable? Why does it say they aren't?

How can I await a Future object returned by executor.submit?

Python 3.5.0

EDIT

Using executor.submit is not my decision. This is used internally by several libraries, like requests-futures. I am searching for a way to interop with those modules from coroutines.

Upvotes: 26

Views: 18068

Answers (1)

Jashandeep Sohi
Jashandeep Sohi

Reputation: 5163

You should use loop.run_in_executor:

from concurrent.futures import ThreadPoolExecutor
import asyncio

def work():
  # do some blocking io
  pass

async def main(loop):
  executor = ThreadPoolExecutor()
  await loop.run_in_executor(executor, work)

loop = asyncio.get_event_loop()
loop.run_until_complete(main(loop))
loop.close()

EDIT

concurrent.futures.Future object are different from asyncio.Future. The asyncio.Future is intended to be used with event loops and is awaitable, while the former isn't. loop.run_in_executor provides the necessary interoperability between the two.

EDIT #2

Using executor.submit is not my decision. This is used internally by several libraries, like requests-futures. I am searching for a way to interop with those modules from coroutines.

EDIT #3

Since python 3.5 (docs), you can use asyncio.wrap_future(future, *, loop=None) to convert a concurrent.futures.Future to a asyncio.Future.

Upvotes: 39

Related Questions