CODEWITHSUNDEEP
asynchronousasync-awaitpython-asyncioaiohttp
Maverick Sachin
Maverick Sachin

Reputation: 883

Output of one request used as a parameter in next request using Asyncio + Aiohttp

I'm new to Async programming, Asyncio, Aiohttp world.

I have multiple requests where - output of 1st request is being used as a parameter in 2nd request.

So far I have tried this much -

import aiohttp
import asyncio


async def get_response(session, url, params):
    async with session.get(url=url, params=params) as response:
        response = await response.json()
        return response['output']


async def main():

    tasks = []

    url = "https://example.api.com/"

    url_1 = url + 'path_1'
    url_2 = url + 'path_2'

    params = {'name': "Hello"}

    async with aiohttp.ClientSession() as session:

        a1 = get_response(session, url_1, params)
        tasks.append(a1)

        params = {'name': a1}
        b1 = get_response(session, url_2, params)
        tasks.append(b1)

        responses = await asyncio.gather(*tasks, return_exceptions=True)

        print(responses)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

I'm getting this error which is obvious -

TypeError("Invalid variable type: value should be str, int or float, got <coroutine object get_response at 0x00000000037FA6C0> of type <class 'coroutine'>")

So how do I pass the output of 1st request and not the coroutine object to the 2nd request -

params = {'name': a1}
b1 = get_response(session, url_2, params)

Upvotes: 0

Views: 832

Answers (1)

user4815162342
user4815162342

Reputation: 154876

If the result of the first coroutine is needed for the second one, then you cannot run both coroutines in parallel, you will need to await the response of the first one. For example:

async def get_both(session, url_1, url_2, params_1):
    a1 = await get_response(session, url_1, params_1)
    params_2 = {'name': a1}
    b1 = await get_response(session, url_2, params_2)
    return a1, b1  # or just b1 if you don't need a1

Your main() would then look like this:

async def main():
    url = "https://example.api.com/"
    url_1 = url + 'path_1'
    url_2 = url + 'path_2'
    params = {'name': "Hello"}

    async with aiohttp.ClientSession() as session:
        responses = await get_both(session, url_1, url_2, params)
        print(responses)

Of course, you don't get parallel fetching, but that's a direct consequence of the fact that one request depends on the result of the other. What you could easily do is parallelize multiple instances of get_both - e.g. if url_1 and url_2 were lists, you could do this:

async def main():
    url_list_1 = [...]
    url_list_2 = [...]
    params = {'name': "Hello"}

    async with aiohttp.ClientSession() as session:
        coros = [get_both(session, url_1, url_2, params)
                 for url_1, url_2 in zip(url_list_1, url_list_2)]
        all_responses = await asyncio.gather(*coros)
        print(all_responses)

Upvotes: 2

Related Questions