CODEWITHSUNDEEP
pythonpython-3.xasync-awaitpython-asyncio
Bite code
Bite code

Reputation: 596713

How to not await in a loop with asyncio?

Here is a toy example that downloads the home page from several websites using asyncio and aiohttp:

import asyncio
import aiohttp

sites = [
    "http://google.com",
    "http://reddit.com",
    "http://wikipedia.com",
    "http://afpy.org",
    "http://httpbin.org",
    "http://stackoverflow.com",
    "http://reddit.com"
]


async def main(sites):
    for site in sites:
        download(site)


async def download(site):
    response = await client.get(site)
    content = await response.read()
    print(site, len(content))


loop = asyncio.get_event_loop()
client = aiohttp.ClientSession(loop=loop)
content = loop.run_until_complete(main(sites))
client.close()

If I run it, I get:

RuntimeWarning: coroutine 'download' was never awaited

But I don't want to await it.

In twisted I can do:

for site in sites:
    download(site)

And If I don't explicitly "yield" or add a callback to the returned Deferred, it just runs without blocking nor complaining. I can't access the result, but in this case I don't need it.

In JS I can do:

site.forEarch(site){
    donwload(site)
}

And again, it doesn't block nor does it requires anything from my part.

I found a way to do:

async def main(sites):
    await asyncio.wait([download(site) for site in sites])

But:

It there a better way ?

Upvotes: 18

Views: 22758

Answers (2)

Jashandeep Sohi
Jashandeep Sohi

Reputation: 5163

  • this is really not obvious to find it out. I it's hard to remember.

The documentation on coroutines does make it pretty clear what asyncio.wait's purpose is.

  • it's hard to understand what it does. "waits" seems to say "i block", but does not convey clearly it block for the entire list of coroutine to finish.

Again, see the documentation.

  • you can't pass in a generator, it needs to be a real list, which i feels really unatural in Python.

Again, see the documentation, specifically asyncio.as_completed

  • what if I have only ONE awaitable ?

It should still work.

  • what if I don't want to wait at all on my tasks, and just schedule them for execution then carry on with the rest of my code ?

Then you can use asyncio.ensure_furture. In fact, asyncio.wait is a convenience function around asyncio.ensure_future (and some other logic).

  • it's way more verbose thant twisted and JS solution.

Maybe, but that's not a bad thing (from my perspective).

Upvotes: -12

Vincent
Vincent

Reputation: 13415

In order to schedule a coroutine as a task, use asyncio.ensure_future:

for site in sites:
    coro = download(site)
    future = asyncio.ensure_future(coro)

It replaces the deprecated function asyncio.async in version 3.4.4.

Then you can manage those futures using await, asyncio.wait or asyncio.gather.

Upvotes: 17

Related Questions