"RuntimeError: This event loop is already running" when combine asyncio and aiohttp

Question

Unlike such post, my scenario is next:

• I already have an asyncio program works well, something like next:

import asyncio

async def action():
    print("action")
    # use sleep to simulate real operation
    # it's a daemon works endless there
    await asyncio.sleep(10000)

async def main():
    await action()

if __name__ == '__main__':
    asyncio.run(main())

• Then, I want to add a rest api for my program. With this, client could query some status of my program with api call(E.g. get some internal status variable of my asyncio program logic):

import asyncio
from aiohttp import web

async def handle(request):
    name = request.match_info.get('name', "Anonymous")
    text = "Hello, " + name
    return web.Response(text=text)

async def start_api_server():
    print("api")
    app = web.Application()
    app.add_routes([web.get('/', handle),
                    web.get('/{name}', handle)])
    web.run_app(app)

async def action():
    print("action")
    await asyncio.sleep(10000)

async def main():
    #await action()
    await(asyncio.gather(start_api_server(), action()))

if __name__ == '__main__':
    asyncio.run(main())

But, above reports next stack:

Traceback (most recent call last):
  File "svr.py", line 26, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete
    return future.result()
  File "svr.py", line 23, in main
    await(asyncio.gather(start_api_server(), action()))
  File "svr.py", line 15, in start_api_server
    web.run_app(app)
  File "/home/cake/venv/lib/python3.7/site-packages/aiohttp/web.py", line 512, in run_app
    _cancel_tasks({main_task}, loop)
  File "/home/cake/venv/lib/python3.7/site-packages/aiohttp/web.py", line 444, in _cancel_tasks
    asyncio.gather(*to_cancel, loop=loop, return_exceptions=True)
  File "/usr/lib/python3.7/asyncio/base_events.py", line 566, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.7/asyncio/base_events.py", line 521, in run_forever
    raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running

Question:

It looks we can't do above combinaton, so I wonder what's the correct way to add a rest api to a asyncio program?

NOTE: if possible, I want to avoid run another thread to start web server unless it's totall impossible.

Answer 1

I find the answer, from aiohttp source code:

• web.run_app(app) works as next:

async def _run_app(......):

    runner = AppRunner(
        app,
        handle_signals=handle_signals,
        access_log_class=access_log_class,
        access_log_format=access_log_format,
        access_log=access_log,
    )

    await runner.setup()
    ......
    while True:
        await asyncio.sleep(delay)

def run_app(......):

    loop = asyncio.get_event_loop()
    try:
        main_task = loop.create_task(
            _run_app(
                app,
                ......
            )
        )
        loop.run_until_complete(main_task)

But it will directly call loop.run_until_complete which makes us not possible for us to start our own event loop which used by asyncio.run() in main program.

• To conquer this, we should use some low-level functions as next:

import asyncio
from aiohttp import web


async def handle(request):
    name = request.match_info.get('name', "Anonymous")
    text = "Hello, " + name
    return web.Response(text=text)

async def start_api_server():
    print("api")
    loop = asyncio.get_event_loop()
    app = web.Application()
    app.add_routes([web.get('/', handle),
                    web.get('/{name}', handle)])
    #web.run_app(app)
    runner = web.AppRunner(app)
    await runner.setup()
    await loop.create_server(runner.server, '127.0.0.1', 8080)
    print('Server started at http://127.0.0.1:8080...')

async def action():
    print("action")
    await asyncio.sleep(10000)

async def main():
    #await action()
    await(asyncio.gather(start_api_server(), action()))

if __name__ == '__main__':
    asyncio.run(main())

Above code reproduce the runner setup in web.run_app, and pass the property server in AppRunner to asyncio.create_server to new a aiohttp server. Then could reuse the eventloop in asyncio.run to meet my requirement.