CODEWITHSUNDEEP
pythonmultithreadingpython-3.xpython-asyncio
Havnar
Havnar

Reputation: 2628

how to use aiohttp in an asycio background thread

I'm trying to write a discord bot that updates users with data I fetch from a http rest call.

Since discord.py uses asyncio, I thought I'd try following this approach through.

async def my_background_task():

    print("starting BG task")
    await client.wait_until_ready()
    while not client.is_closed:
        requests.get(getMyUrl()).json()["dict_element"]
        #do more stuff

client.loop.create_task(my_background_task())
client.run("api key goes here")

It's straight forward enough using the sync "request" library. However I can run this for about a few hours until it crashes. I assume because of 'request' failing to finish the endless loop.

async def fetch(session, url):
    async with session.get(url) as resp:
        print(resp.status)
        return await resp

async def fetch_url(loop, url):
    async with aiohttp.ClientSession(loop=loop) as session:
        await fetch(session, url)

async def my_method(url):

    loop = asyncio.get_event_loop()
    return loop.run_until_complete(fetch_url(loop,url))

async def my_background_task():

    print("starting BG task")
    await client.wait_until_ready()
    while not client.is_closed:
        await my_method.("www.theinternet.com/restcall").json()["dict_element"]
        #do more stuff

So I now try to work with aiohttp but run into issues with async loops within this loop.

I haven't found a proper explanation on how to approach this. I'm pretty new to python and async functions.

Upvotes: 0

Views: 1843

Answers (2)

Arthur
Arthur

Reputation: 4241

Without the error you get from your aiohttp version, it's hard to guess. But at first sight, I would say that :

  • No need to pass the event loop explicitly to ClientSession, it will automatically choose the current event loop, so you should just call ClientSession()
  • the fetch function should not await resp. resp is not a coroutine there. You probably want to return await resp.json() (and remove the extra .json() in my_background_task)
  • In my_method, you are already running in a coroutine, as indicated by the async def. So to call another coroutine, you should be await-ing it, not creating a new event loop and running it in that new loop.
  • In my_background_task, you are awaiting something that does not make any sense: check the operator priority of await, you are calling .__getitem__ on a coroutine, and then awaiting the result of __getitem__, which is the reverse of what you want.
  • And still on that line, you cannot try to read the json body of the response after the response has been closed (the response is closed as soon as you exit the async with, ie as soon as the fetch method is done).
  • Finally, you should not be re-creating a ClientSession at every loop. This exists so that it can be shared between requests, so you should create it once in my_background_task and use it

-

async def fetch(session, url):
    async with session.get(url) as resp:
        print(resp.status)
        return await resp.json()

async def my_background_task()
    print("starting BG task")
    await client.wait_until_ready()
    async with aiohttp.ClientSession() as session:
        while not client.is_closed:
            data = await fetch(session, "www.theinternet.com/restcall")
            element = data["dict_element"]

Upvotes: 1

Havnar
Havnar

Reputation: 2628

I got around this by having another background task that keeps the http response as a global variable and updates it on its own.

I'm no sure if that's the way I'm supposed to do it, but that's what got me out of a pickle.

Upvotes: 0

Related Questions