CODEWITHSUNDEEP
python-3.xmultithreadingpython-asynciovalueerror
Axel Paccalin
Axel Paccalin

Reputation: 133

ValueError when asyncio.run() is called in separate thread

I have a network application which is listening on multiple sockets. To handle each socket individually, I use Python's threading.Thread module.

These sockets must be able to run tasks on packet reception without delaying any further packet reception from the socket handling thread.

To do so, I've declared the method(s) that are running the previously mentioned tasks with the keyword async so I can run them asynchronously with asyncio.run(my_async_task(my_parameters)).

I have tested this approach on a single socket (running on the main thread) with great success. But when I use multiple sockets (each one with it's independent handler thread), the following exception is raised:

ValueError: set_wakeup_fd only works in main thread

My question is the following: Is asyncio the appropriate tool for what I need? If it is, how do I run an async method from a thread that is not a main thread.

Most of my search results are including "event loops" and "awaiting" assync results, which (if I understand these results correctly) is not what I am looking for.

I am talking about sockets in this question to provide context but my problem is mostly about the behaviour of asyncio in child threads.

I can, if needed, write a short code sample to reproduce the error. Thank you for the help!

Edit1, here is a minimal reproducible code example:

import asyncio
import threading
import time


# Handle a specific packet from any socket without interrupting the listenning thread
async def handle_it(val):
    print("handled: {}".format(val))  


# A class to simulate a threaded socket listenner
class MyFakeSocket(threading.Thread):
    def __init__(self, val):
        threading.Thread.__init__(self)
        self.val = val  # Value for a fake received packet

    def run(self):
        for i in range(10):
            # The (fake) socket will sequentially receive [val, val+1, ... val+9]
            asyncio.run(handle_it(self.val + i))
            time.sleep(0.5)


# Entry point
sockets = MyFakeSocket(0), MyFakeSocket(10)
for socket in sockets:
    socket.start()

Upvotes: 1

Views: 642

Answers (1)

PirateNinjas
PirateNinjas

Reputation: 2076

This is possibly related to the bug discussed here: https://bugs.python.org/issue34679

If so, this would be a problem with python 3.8 on windows. To work around this, you could try either downgrading to python 3.7, which doesn't include asyncio.main so you will need to get and run the event loop manually like:

loop = asyncio.get_event_loop()
loop.run_until_complete(<your tasks>)
loop.close()

Otherwise, would you be able to run the code in a docker container? This might work for you and would then be detached from the OS behaviour, but is a lot more work!

Upvotes: 1

Related Questions