CODEWITHSUNDEEP
pythonpython-asyncio
trogne
trogne

Reputation: 3562

asyncio not stopping where expected

With this asyncio code, my tutor says that all coroutines (here the 50 "fetch_page") first stop at the first async with and wait, then all of them resume from there and stop at the second async with, then finally all of them all return.

import aiohttp
import asyncio

async def fetch_page(url):
    print(1)
    async with aiohttp.ClientSession() as session:
       print(2)
        async with session.get(url) as response:
            print(3)
            return response.status

loop = asyncio.get_event_loop()    
tasks = [fetch_page('http://google.com') for i in range(50)]
loop.run_until_complete(asyncio.gather(*tasks))

I'm debugging this, and I must say he's wrong. While debugging, I see all coroutines going sequentially to the second async with, where they all stop. Then once all the 50 coroutines resumes, they do the session.get(url) and return.

But why not all of the couroutines stop at the first async with ?

Print output : "1 2 1 2 1 2 ... 3 3 3 ...", instead of "1 1 1 ... 2 2 2 ... 3 3 3 ..."

Upvotes: 1

Views: 352

Answers (2)

Ramy M. Mousa
Ramy M. Mousa

Reputation: 5943

There is a little piece of info is missing here:

asyncio is a single-threaded library based on event loop.

What happens in this case is that aiohttp.ClientSession() as session is not an I/O expensive operation, it doesn't take time to execute, that's why it gets executed immediately even before the iterator gets to the next loop cycle.

Why 3333 get's printed last? because you're making an I/O that takes time, more than looping for 50 times.

For more details read here: asyncio python doc

Upvotes: 1

user4815162342
user4815162342

Reputation: 155515

But why not all of the couroutines stop at the first async with?

Just like any other await, an async with doesn't guarantee that the coroutine will suspend there, but it allows the coroutine to suspend if there is reason to do so. In this case, merely creating a ClientSession is an operation that can be done without suspending any coroutine, so each of them simply proceeds further.

Entering the second async with must obtain a response object, which requires that the request has been sent and the HTTP headers received. Executed on a non-blocking socket, some of these operations will signal that the data is not immediately available (because it needs to reach the server, and the server has to craft a response, and then the response has to travel back to us), and asyncio handles that by suspending the coroutine. This is why the second async with is practically guaranteed to suspend each coroutine that reaches it.

Upvotes: 2

Related Questions