CODEWITHSUNDEEP
python-asynciononblocking
Andrey Paramonov
Andrey Paramonov

Reputation: 409

Custom non-blocking operations in Python asyncio

Currently, asyncio Event Loop supports non-blocking work with network sockets, files, subprocesses, and signals. For other operations, documentations describes executing code in thread or process pools, but that seems much less efficient (threads vs coroutines).

Is there a way to introduce additional low-level non-blocking operations, e.g. wrapping I/O-heavy library calls? What primitives must be implemented?

Upvotes: 1

Views: 720

Answers (1)

user4815162342
user4815162342

Reputation: 154916

Like most event loops, the asyncio event loop is built around polling IO sources, file descriptors on Unix and file handles on Windows. Poll or select is an IO operation that efficiently monitors multiple file descriptors, suspending the current thread until something interesting happens, e.g. new data arrives. Optionally, polling accepts a timeout, so execution can continue even if there is no new IO event. For a more detailed treatment of the topic, refer to this SO answer and other answers to the same question.

The poll-based design allows asyncio to natively support two kinds of events:

  • IO possible
  • timeout expired

All other types of events must be expressed in terms of those two. For example, call_soon_threadsafe wakes up the event loop by writing into an internal pipe monitored by the event loop. The implementation of run_in_executor uses call_soon_threadsafe to inform asyncio that the synchronous function has completed.

To connect an entirely different polling mechanism to asyncio, one would typically spawn a thread dedicated to that kind of polling, and use call_soon_threadsafe to inform asyncio of new events. This answer, for example, shows how to connect the raw multiprocessing pool (which can be terminated) to asyncio.

Upvotes: 1

Related Questions