How to call async code from sync code in another thread?

Question

I'm making a Discord bot which send PM when it receive a Github hook.

It use Discord.py and BottlePy, the last one run in a dedicated thread. Because both frameworks have a blocking main loop.

In BottlePy callback, I call some Discord.py async code.

I wasn't knowing what is Python async, this appear to be complicated when mixed with synchronous code...

Here's the full source code :

import discord
import bottle
import threading
import asyncio

client = discord.Client()
server = bottle.Bottle()

async def dm_on_github_async(userid,request):
    print("Fire Discord dm to "+str(userid))
    global client
    user = client.get_user(userid)
    if (user==None):
        abort(500, "User lookup failed");

    dm_channel = user.dm_channel
    if (dm_channel==None):
        dm_channel = await user.create_dm()
    if (dm_channel==None):
        abort(500, "Fail to create DM channel");
    print("DM channel is "+str(asyncio.wait(dm_channel.id)))
    await dm_channel.send("There's a Github shot !")
    await dm_channel.send(str(request.body))
    return
@server.post("/dm_on_github/<userid:int>")
def dm_on_github(userid):
    return asyncio.run(dm_on_github_async(userid,bottle.request))
@client.event
async def on_ready():
    print('We have logged in as {0.user} '.format(client))

#@client.event
#async def on_message(message):
#    if message.author == client.user:
#        return
#
#    if message.content.startswith('$hello'):
#        await message.channel.send('Hello!')
#    # This sample was working very well

class HTTPThread(threading.Thread):
    def run(self):
        global server
        server.run(port=8080)
server_thread = HTTPThread()
print("Starting HTTP server")
server_thread.start()
print("Starting Discord client")
client.run('super secret key')
print("Client terminated")
server.close()
print("Asked server to terminate")
server_thread.join()
print("Server thread successful join")

I want that my Python bot send the body of the HTTP request as PM.

I get a RuntimeError: Timeout context manager should be used inside a task at return asyncio.run(dm_on_github_async(userid,bottle.request)).

I think I'm not doing this mix in the right way...

Answer 1

After a night, I found the way.

To call async code from sync code in another thread, we ask the loop (here this one from Discord.py) to run the callback with asyncio.run_coroutine_threadsafe(), this return a Task() and we wait for his result with result().

The callback will be run in the loop thread, in my case I need to copy() the Bottle request.

Here's a working program (as long you don't mind to stop it...) :

import discord
import bottle
import threading
import asyncio

client = discord.Client()
server = bottle.Bottle()
class HTTPThread(threading.Thread):
    def run(self):
        global server
        server.run(port=8080)

async def dm_on_github_async(userid,request):
    user = client.get_user(userid)
    if (user==None):
        abort(500, "User lookup failed");
    dm_channel = user.dm_channel
    if (dm_channel==None):
        dm_channel = await user.create_dm()
    if (dm_channel==None):
        abort(500, "Fail to create DM channel");
    # Handle the request
    event = request.get_header("X-GitHub-Event")
    await dm_channel.send("Got event "+str(event))
    #await dm_channel.send(str(request.body)) # Doesn't work well...
    return

@server.post("/dm_on_github/<userid:int>")
def dm_on_github(userid):
    request = bottle.request
    asyncio.run_coroutine_threadsafe(dm_on_github_async(userid,request.copy()),client.loop).result()


@client.event
async def on_ready():
    print('We have logged in as {0.user} '.format(client))
    # Wait for the old HTTP server
    if hasattr(client,"server_thread"):
        server.close()
        client.server_thread.join()
    client.server_thread = HTTPThread()
    client.server_thread.start()

#@client.event
#async def on_message(message):
#    if message.author == client.user:
#        return
#
#    if message.content.startswith('$hello'):
#        await message.channel.send('Hello!')

print("Starting Discord client")
client.run('super secret key')
print("Client terminated")
server.close()
print("Asked server to terminate")
server_thread.join()
print("Server thread successful join")