CODEWITHSUNDEEP
python-3.xpython-asyncio
Spark
Spark

Reputation: 115

how to get return value of async coroutine python

I am new to python and trying to learn asyncio module. I am frustrated on getting return values from async tasks. There is a post here talked about this topic, but it can't tell which value is returned by which task(assuming some one web page response faster than another).
The code below is trying to fetch three web pages concurrently instead of doing it one by one.

    import asyncio
    import aiohttp

    async def fetch(url):
        async with aiohttp.ClientSession() as session:
            async with session.get(url) as resp:
                assert resp.status == 200
                return await resp.text()

    def compile_all(urls):
        tasks = []
        for url in urls:
            tasks.append(asyncio.ensure_future(fetch(url)))
        return tasks

    urls = ['https://python.org', 'https://google.com', 'https://amazon.com']
    tasks = compile_all(urls)
    loop = asyncio.get_event_loop()
    a, b, c = loop.run_until_complete(asyncio.gather(*tasks))
    loop.close()

    print(a)
    print(b)
    print(c)

First, it hit Runtimeerror though it did print some html documents: RuntimeError: Event loop is closed.

Second, question is: does this really guarantee that a, b, c will be corresponded to the urls list in sequence of urls[0], url[1], urls[2] web page? (I assume that async tasks execution won't guarantee that).

Third, any other better means or Should I use Queue in this case? if yes, how?

Any help will be greatly appreciated.

Upvotes: 1

Views: 9396

Answers (1)

countunique
countunique

Reputation: 4296

The order of the results will correspond to the order of the urls. Take a look at the docs for asyncio.gather:

If all awaitables are completed successfully, the result is an aggregate list of returned values. The order of result values corresponds to the order of awaitables in aws.

To process tasks as they complete you can use asyncio.as_completed. This post has more information on how it can be used.

Upvotes: 2

Related Questions