CODEWITHSUNDEEP
pythonasynchronouspython-asyncio
Johnemperor
Johnemperor

Reputation: 96

While loop in async function throws StopIteration

I am working on program, which sends a bunch of requests to one service asynchronously. Since now i had the part, where there was just request and awaiting for results (ok_function). Now I needed to add part which gets result, and if it is 202 waits for some time and try again since it return 200.

It looks like this:

async def async_make_requests(parsed_data, url):
    async with aiohttp.ClientSession() as session:
        tasks = []
        for x, y in parsed_data.items():

            if something:
                tasks.append(
                    asyncio.ensure_future(
                        function_with_sleep(
                            session, x, y,
                        )
                    )
                )
             else:
                 tasks.append(
                    asyncio.ensure_future(
                        ok_function(
                            session, x, y,
                        )
                    )
                )   
        results = await asyncio.gather(*tasks)
    return results

async def function_with_sleep(session, x, y):
    not_processed_yet = True
    while not_processed_yet:
        async with session.get(x, data=y) as r:
            response = await r.json()
            status_code = r.status
        if status_code == 202:
            await asyncio.sleep(10)
        else:
            not_processed_yet = False
   ...

async def ok_function(session, x, y)
    async with session.post(x, data=y) as r:
        response = await r.json()
        status_code = r.status
    ...

While testing it with:

    resp_processing = MockResponse(202, response)
    resp_ok = MockResponse(200, response)
    mocker.patch("aiohttp.ClientSession.get", side_effect=[resp_processing, resp_ok])
    return_value = await async_make_requests(parsed_data, "anyurl")

I am getting:

  results = await asyncio.gather(*tasks) RuntimeError: coroutine raised StopIteration

ok_function works fine, and function_with_sleep works ok only if it doesn't repeat itself, and doesn't sleep.

I have no idea what is wrong here.

Upvotes: 1

Views: 1900

Answers (1)

jsbueno
jsbueno

Reputation: 110271

The code seens to be right - although you don't say which http framework your session belongs too: If r.json raises an error when there is no json body in the request made, this error would be propagated - and if it is StopIteration, it will break asyncIO.gather as described (otherwise, the task exception would be set in the corresponding task).

Try to call await r.json() just after you have a 200 status code, and not before testing the status code. If the behavior you claim is indeed caused by the code you pasted in the question, this should fix it.

Indeed - it seems you are using aiohttp, or something that builds upon it: if a body is not available in the request object, it will call .read in the response - since the response is a mock object this is likely triggering the StopIteration exception. - https://docs.aiohttp.org/en/stable/client_reference.html

Trying to get the JSON body just when you have a meaningful response is the right thing to do, nonetheless, so there is no point in trying to add a correct read response to your mock.

Upvotes: 2

Related Questions