Why you cannot await a python coroutine object directly?

Question

So I am running an asyncio example:

import asyncio, time

async def say_after(delay, what):
    await asyncio.sleep(delay)
    print(what)

async def main():
    task1 = asyncio.create_task(say_after(1, 'hello'))

    task2 = asyncio.create_task(say_after(2, 'world'))

    print(f"started at {time.strftime('%X')}")

    # Wait until both tasks are completed (should take
    # around 2 seconds.)
    await task1
    await task2

    print(f"finished at {time.strftime('%X')}")

asyncio.run(main())

This piece of code works correctly with output:

started at 14:36:06
hello
world
finished at 14:36:08

The 2 coroutines is running asynchronously, finally took 2 seconds, which has no problem. However, when I combine the lines together and directly await the Task object, like this:

import asyncio, time

async def say_after(delay, what):
    await asyncio.sleep(delay)
    print(what)

async def main():

    print(f"started at {time.strftime('%X')}")

    # Wait until both tasks are completed (should take
    # around 2 seconds.)
    await asyncio.create_task(say_after(1, 'hello'))
    await asyncio.create_task(say_after(2, 'world'))

    print(f"finished at {time.strftime('%X')}")

asyncio.run(main())

This result becomes:

started at 14:37:12
hello
world
finished at 14:37:15

which took 3 seconds, indicating that coroutine running incorrectly.

How can I make the latter code working properly? or is there something idk resulting in this difference?

P.S. The example acturally comes from python doc: https://docs.python.org/3.8/library/asyncio-task.html#coroutines

Answer 1

await makes it so that code 'stops' and continues after the awaited coroutine is completed, so when you write

await asyncio.create_task(say_after(1, 'hello'))
await asyncio.create_task(say_after(2, 'world'))

the second task is created and run after the first coroutine was completed, therefore it takes 3 seconds total. As a solution, consider using a function like gather or wait. For example:

    import asyncio, time
    
    async def say_after(delay, what):
        await asyncio.sleep(delay)
        print(what)
    
    async def main():
        print(f"started at {time.strftime('%X')}")

        # Wait until both tasks are completed (should take
        # around 2 seconds.)
        await asyncio.gather(say_after(1, 'hello'), say_after(2, 'world'))

        print(f"finished at {time.strftime('%X')}")

    asyncio.run(main())

Output:

started at 08:10:04
hello
world
finished at 08:10:06

Answer 2

From the docs Await expression:

Suspend the execution of coroutine on an awaitable object. Can only be used inside a coroutine function.

Whenever you await, the routine is suspended until the waited task completes. In the first example, both coroutines start and the 2 second sleep in the second overlaps the first. By the time you start running after the first await, 1 second has already elapsed in the second timer.

In the second example, the second await asyncio.create_task(say_after(2, 'world')) isn't scheduled until after the first completes and main continues running. That's when the 2 second sleep for the second task begins.

I've combined the examples to show the progression. Instead of the original prints, I print a start message before say_after awaits and a finish message just after main's await. You can see the time difference in the results.

import asyncio, time

async def say_after(delay, what):
    print(f"start {what} at {time.strftime('%X')}")
    await asyncio.sleep(delay)
    print(what)

async def main():
    task1 = asyncio.create_task(say_after(1, 'hello'))
    task2 = asyncio.create_task(say_after(2, 'world'))
    await task1
    print(f"Finished hello at {time.strftime('%X')}")
    await task2
    print(f"Finished world at {time.strftime('%X')}")

async def main2():
    await asyncio.create_task(say_after(1, 'hello'))
    print(f"Finished hello at {time.strftime('%X')}")
    await asyncio.create_task(say_after(2, 'world'))
    print(f"Finished world at {time.strftime('%X')}")

print("========== Test 1 ============")
asyncio.run(main())

print("========== Test 2 ============")
asyncio.run(main2())

The results of the second test show that the second say_after isn't called until the first completes.

========== Test 1 ============
start hello at 00:51:42
start world at 00:51:42
hello
Finished hello at 00:51:43
world
Finished world at 00:51:44
========== Test 2 ============
start hello at 00:51:44
hello
Finished hello at 00:51:45
start world at 00:51:45
world
Finished world at 00:51:47

In main, tasks are created to run asyncio.sleep, but those tasks aren't actually run until main returns to the even loop. If we add a time.sleep(3) we might expect these two overlapped async sleeps to already be complete, but in fact say_after isn't even run until the first await that lets the event loop continue.

import asyncio, time

async def say_after(delay, what):
    print(f"starting {what} at {time.time()-start}")
    await asyncio.sleep(delay)
    print(what)

async def main():
    global start
    print('time asyncio.sleep with intermedite time.sleep')
    start = time.time()
    task1 = asyncio.create_task(say_after(1, 'hello'))
    task2 = asyncio.create_task(say_after(2, 'world'))

    # similate working for 3 seconds with non asyncio sleep
    time.sleep(3)
    print(f'expect 3 got {time.time()-start}')
    await task1  # <== where the 2 `say_after` tasks start
    print(f'expect 3 got {time.time()-start}')
    await task2
    print(f'expect 3 got {time.time()-start}')

asyncio.run(main())

Produces

time asyncio.sleep with intermedite time.sleep
expect 3 got 3.0034446716308594
starting hello at 3.003699541091919
starting world at 3.0038907527923584
hello
expect 3 got 4.005880355834961
world
expect 3 got 5.00671124458313

Adding an asyncio.sleep(0) to main after setting up the tasks allows them to run and do their own overlapped sleeps and the code works as we want.

import asyncio, time

async def say_after(delay, what):
    print(f"starting {what} at {time.time()-start}")
    await asyncio.sleep(delay)
    print(what)

async def main():
    global start
    print('time asyncio.sleep with event loop poll and intermedite time.sleep')
    start = time.time()
    task1 = asyncio.create_task(say_after(1, 'hello'))
    task2 = asyncio.create_task(say_after(2, 'world'))

    # let the `say_after` tasks (and anything else pending) run
    await asyncio.sleep(0)

    # similate working for 3 seconds with non asyncio sleep
    time.sleep(3)
    print(f'expect 3 got {time.time()-start}')
    await task1  # <== where the 2 `say_after` tasks start
    print(f'expect 3 got {time.time()-start}')
    await task2
    print(f'expect 3 got {time.time()-start}')

asyncio.run(main())

Answer 3

I kinda understand the question now...

await makes the process blocked at that line.

So in the main function, if you want to do parrellel tasks, better use asyncio.wait/gather...

I think it is just the design style of Asyncio which makes the former code working well...