CODEWITHSUNDEEP
pythonpython-asyncio
anjuna47
anjuna47

Reputation: 95

asyncio.create_task without awaiting result

I am trying to make sense of asyncio, specifically the await keyword.

This is my code:

async def sleep_and_print(s: str):

   print(f'{datetime.utcnow()} Executing sleep_and_print({s})')
   await asyncio.sleep(1)
   print(f'{datetime.utcnow()} Completed sleep_and_print({s})')

async def main():

   asyncio.create_task(sleep_and_print(0))
   asyncio.create_task(sleep_and_print(1))
   asyncio.create_task(sleep_and_print(2))

   asyncio.gather(*[sleep_and_print(i) for i in range(3)])

if __name__ == "__main__":

   asyncio.run(main())

The output is:

2021-10-14 10:08:03.242891 Executing sleep_and_print(0)
2021-10-14 10:08:03.242891 Executing sleep_and_print(1)       
2021-10-14 10:08:03.242891 Executing sleep_and_print(2)       
2021-10-14 10:08:03.243896 Executing sleep_and_print(0)       
2021-10-14 10:08:03.243896 Executing sleep_and_print(1)       
2021-10-14 10:08:03.243896 Executing sleep_and_print(2)       
_GatheringFuture exception was never retrieved
future: <_GatheringFuture finished exception=CancelledError()>
concurrent.futures._base.CancelledError

I have two initial questions:

  1. Why is the behaviour different when creating three tasks with create_task() compared with the tasks being created by gather()? I understand that gather() will submit the tasks for execution concurrently whereas the create_task() tasks will be submitted individually. I do not understand why gather() result in an exception.

  2. Why is it that this task will begin but not complete execution

    asyncio.create_task(sleep_and_print(0))

    Whereas both of these will complete, even though only the second is awaited on

    asyncio.create_task(sleep_and_print(0))
await asyncio.create_task(sleep_and_print(1))

Upvotes: 4

Views: 5523

Answers (1)

2e0byo
2e0byo

Reputation: 5964

create_task is synchronous: it merely submits the task to the event loop and returns immediately.

Wrap the coro coroutine into a Task and schedule its execution. Return the Task object.

Thus if you await it you just await the task object, thus await create_task(fn()) is functionally equivalent to await fn().

gather() is asynchronous. You have to await it, otherwise you just get the coro it returns, or more accurately, the Futures object. The exception is being thrown when the loop ends, since gather() has never actually been run to completion* (since nothing awaited it).

More generally, the effect of await from your point of view is to pause execution until the result is 'in' and then continue. The effect from the code's point of view is to yield back to the event loop and stop running until the condition awaited is satisfied, at which point the event loop might decide, next time something yields to it and it gets to choose which task to run, to go back to your task and keep running. (It might not as well: remember that asyncio will never continue before the awaited is complete, but continuing straight afterwards is best-effort. Thus if you await asyncio.sleep(0.0001) you will likely end up waiting longer than you expected. This is rarely a problem on real hardware, but it can make writing asynchronous drivers in things like micropython a little harder.)

create task is an idiom for 'run this pretty soon'. (If you've written ISRs, you may have used similar strategies to avoid running too much code in the interrupt context, when you need to run a moderately expensive callback on some event.) Note that if you call create_task from within the event loop and do not yield (by awaiting something) the task will never run to completion*; likewise if the loop ends, it will never run (as here, since run only runs until main is finished).

Note that it is perfectly meaningful to generate a coro and then await it later:

async def my_fn():
   await asyncio.sleep(56)

long_fn = my_fn()
...
await long_fn

You might do this when e.g. passing coros as arguments to functions, or storing them in class attributes. Coros are every bit as much first-class citizens as functions in python.

*@user118967 points out here that all scheduled tasks will in fact be advanced once at the end of the event loop. Thus a coroutine which didn't actually yield would in fact be run to completion at the end of the loop anyway, athough any real coroutine will simply progress to an incomplete state. This behaviour should not be relied on in any sense---it doesn't make sense to exit the loop before everything is done---but it does show that if you want to cancel all tasks you need to get the task objects and call task.cancel on them (which causes them to raise at the next await barrier/checkpoint/yield point). None of this applies to gather in the example, which was never awaited or submitted to create_task.

References

https://docs.python.org/3/library/asyncio-task.html#asyncio.gather

(note the use of awaitable to specify that the object returned needs to be awaited)

P.S.

Note that I use 'yield' here in the coroutine sense of 'give up control to the scheduler' rather than in the generator sense of 'send some value'. There was a time in python when we mixed those two together, but thankfully the days of generator based coroutines are over. It worked, but it got very confusing to follow very quickly.

Upvotes: 4

Related Questions