CODEWITHSUNDEEP
linuxasynchronousposixoverlapped-ioio-uring
Jon Watte
Jon Watte

Reputation: 7198

Is there really no asynchronous block I/O on Linux?

Consider an application that is CPU bound, but also has high-performance I/O requirements.

I'm comparing Linux file I/O to Windows, and I can't see how epoll will help a Linux program at all. The kernel will tell me that the file descriptor is "ready for reading," but I still have to call blocking read() to get my data, and if I want to read megabytes, it's pretty clear that that will block.

On Windows, I can create a file handle with OVERLAPPED set, and then use non-blocking I/O, and get notified when the I/O completes, and use the data from that completion function. I need to spend no application-level wall-clock time waiting for data, which means I can precisely tune my number of threads to my number of cores, and get 100% efficient CPU utilization.

If I have to emulate asynchronous I/O on Linux, then I have to allocate some number of threads to do this, and those threads will spend a little bit of time doing CPU things, and a lot of time blocking for I/O, plus there will be overhead in the messaging to/from those threads. Thus, I will either over-subscribe or under-utilize my CPU cores.

I looked at mmap() + madvise() (WILLNEED) as a "poor man's async I/O" but it still doesn't get all the way there, because I can't get a notification when it's done -- I have to "guess" and if I guess "wrong" I will end up blocking on memory access, waiting for data to come from disk.

Linux seems to have the starts of async I/O in io_submit, and it seems to also have a user-space POSIX aio implementation, but it's been that way for a while, and I know of nobody who would vouch for these systems for critical, high-performance applications.

The Windows model works roughly like this:

  1. Issue an asynchronous operation.
  2. Tie the asynchronous operation to a particular I/O completion port.
  3. Wait on operations to complete on that port
  4. When the I/O is complete, the thread waiting on the port unblocks, and returns a reference to the pending I/O operation.

Steps 1/2 are typically done as a single thing. Steps 3/4 are typically done with a pool of worker threads, not (necessarily) the same thread as issues the I/O. This model is somewhat similar to the model provided by boost::asio, except boost::asio doesn't actually give you asynchronous block-based (disk) I/O.

The difference to epoll in Linux is that in step 4, no I/O has yet happened -- it hoists step 1 to come after step 4, which is "backwards" if you know exactly what you need already.

Having programmed a large number of embedded, desktop, and server operating systems, I can say that this model of asynchronous I/O is very natural for certain kinds of programs. It is also very high-throughput and low-overhead. I think this is one of the remaining real shortcomings of the Linux I/O model, at the API level.

Upvotes: 62

Views: 29679

Answers (4)

Anon
Anon

Reputation: 7144

(2020) If you're using a 5.1 or above Linux kernel you can use the io_uring interface for file-like I/O and obtain excellent asynchronous operation.

Compared to the existing libaio/KAIO interface, io_uring has the following advantages:

  • Retains asynchronous behaviour when doing buffered I/O (and not just when doing direct I/O)
  • Easier to use (especially when using the liburing helper library)
  • Can optionally work in a polled manner (but you'll need higher privileges to enable this mode)
  • Less bookkeeping space overhead per I/O
  • Lower CPU overhead due to fewer userspace/kernel syscall mode switches (a big deal these days due to the impact of spectre/meltdown mitigations)
  • File descriptors and buffers can be pre-registered to save mapping/unmapping time
  • Faster (can achieve higher aggregate throughput, I/Os have a lower latency)
  • "Linked mode" can express dependencies between I/Os (>=5.3 kernel)
  • Can work with socket based I/O (recvmsg()/sendmsg() are supported from >=5.3, see messages mentioning the word support in io_uring.c's git history)
  • Supports attempted cancellation of queued I/O (>=5.5)
  • Can request that I/O always be performed from asynchronous context rather than the default of only falling back to punting I/O to an asynchronous context when the inline submission path triggers blocking (>=5.6 kernel)
  • Growing support for performing asynchronous operations beyond read/write (e.g. fsync (>=5.1), fallocate (>=5.6), splice (>=5.7) and more)
  • Higher development momentum
  • Doesn't become blocking each time the stars aren't perfectly aligned

Compared to glibc's POSIX AIO, io_uring has the following advantages:

The Efficient IO with io_uring document goes into far more detail as to io_uring's benefits and usage. The What's new with io_uring document describes new features added to io_uring between the 5.2 - 5.5 kernels, while The rapid growth of io_uring LWN article describes which features were available in each of the 5.1 - 5.5 kernels with a forward glance to what was going to be in 5.6 (also see LWN's list of io_uring articles). There's also a Faster IO through io_uring Kernel Recipes videoed presentation (slides) from late 2019 and What’s new with io_uring Kernel Recipes videoed presentation (slides) from mid 2022 by io_uring author Jens Axboe. Finally, the Lord of the io_uring tutorial gives an introduction to io_uring usage.

The io_uring community can be reached via the io_uring mailing list and the io_uring mailing list archives show daily traffic at the start of 2021.

Re "support partial I/O in the sense of recv() vs read()": a patch went into the 5.3 kernel that will automatically retry io_uring short reads and a further commit went into the 5.4 kernel that tweaks the behaviour to only automatically take care of short reads when working with "regular" files on requests that haven't set the REQ_F_NOWAIT flag (it looks like you can request REQ_F_NOWAIT via IOCB_NOWAIT or by opening the file with O_NONBLOCK). Thus you can get recv() style- "short" I/O behaviour from io_uring too.

Software/projects using io_uring

Though the interface is young (its first incarnation arrived in May 2019), some open-source software is using io_uring "in the wild":

Software investigating using io_uring

Linux distribution support for io_uring

  • (Late 2020) Ubuntu 18.04's latest HWE enablement kernel is 5.4 so io_uring syscalls can be used. This distro doesn't pre-package the liburing helper library but you can build it for yourself.
  • Ubuntu 20.04's initial kernel is 5.4 so io_uring syscalls can be used. As above, the distro doesn't pre-package liburing.
  • Fedora 32's initial kernel is 5.6 and it has a packaged liburing so io_uring is usable.
  • SLES 15 SP2 has a 5.3 kernel so io_uring syscalls can be used. This distro doesn't pre-package the liburing helper library but you can build it for yourself.
  • (Mid 2021) RHEL 8's default kernel does not support io_uring (a previous version of this answer mistakenly said it did). There is an Add io_uring support Red Hat knowledge base article (contents is behind a subscriber paywall) that is "in progress".
  • (Mid 2022) RHEL 9's default kernel does not support io_uring. The kernel is new enough (5.14) but support for io_uring is explicitly disabled.

Hopefully io_uring will usher in a better asynchronous file-like I/O story for Linux.

(To add a thin veneer of credibility to this answer, at some point in the past Jens Axboe (Linux kernel block layer maintainer and inventor of io_uring) thought this answer might be worth upvoting :-)

Upvotes: 88

Peter Teoh
Peter Teoh

Reputation: 6713

As explained in:

http://code.google.com/p/kernel/wiki/AIOUserGuide

and here:

http://www.ibm.com/developerworks/library/l-async/

Linux does provide async block I/O at the kernel level, APIs as follows:

aio_read    Request an asynchronous read operation
aio_error   Check the status of an asynchronous request
aio_return  Get the return status of a completed asynchronous request
aio_write   Request an asynchronous operation
aio_suspend Suspend the calling process until one or more asynchronous requests have completed (or failed)
aio_cancel  Cancel an asynchronous I/O request
lio_listio  Initiate a list of I/O operations

And if you asked who are the users of these API, it is the kernel itself - just a small subset is shown here:

./drivers/net/tun.c (for network tunnelling):
static ssize_t tun_chr_aio_read(struct kiocb *iocb, const struct iovec *iv,

./drivers/usb/gadget/inode.c:
ep_aio_read(struct kiocb *iocb, const struct iovec *iov,

./net/socket.c (general socket programming):
static ssize_t sock_aio_read(struct kiocb *iocb, const struct iovec *iov,

./mm/filemap.c (mmap of files):
generic_file_aio_read(struct kiocb *iocb, const struct iovec *iov,

./mm/shmem.c:
static ssize_t shmem_file_aio_read(struct kiocb *iocb,

etc.

At the userspace level, there is also the io_submit() etc API (from glibc), but the following article offer an alternative to using glibc:

http://www.fsl.cs.sunysb.edu/~vass/linux-aio.txt

It directly implement the API for functions like io_setup() as direct syscall (bypassing glibc dependencies), a kernel mapping via the same "__NR_io_setup" signature should exist. Upon searching the kernel source at:

http://lxr.free-electrons.com/source/include/linux/syscalls.h#L474 (URL is applicable for the latest version 3.13) you are greeted with the direct implementation of these io_*() API in the kernel:

474 asmlinkage long sys_io_setup(unsigned nr_reqs, aio_context_t __user *ctx);
475 asmlinkage long sys_io_destroy(aio_context_t ctx);
476 asmlinkage long sys_io_getevents(aio_context_t ctx_id,
481 asmlinkage long sys_io_submit(aio_context_t, long,
483 asmlinkage long sys_io_cancel(aio_context_t ctx_id, struct iocb __user *iocb,

The later version of glibc should make these usage of "syscall()" to call sys_io_setup() unnecessary, but without the latest version of glibc, you can always make these call yourself if you are using the later kernel with these capabilities of "sys_io_setup()".

Of course, there are other userspace option for asynchronous I/O (eg, using signals?):

http://personal.denison.edu/~bressoud/cs375-s13/supplements/linux_altIO.pdf

or perhap:

What is the status of POSIX asynchronous I/O (AIO)?

"io_submit" and friends are still not available in glibc (see io_submit manpages), which I have verified in my Ubuntu 14.04, but this API is linux-specific.

Others like libuv, libev, and libevent are also asynchronous API:

http://nikhilm.github.io/uvbook/filesystem.html#reading-writing-files

http://software.schmorp.de/pkg/libev.html

http://libevent.org/

All these API aimed to be portable across BSD, Linux, MacOSX, and even Windows.

In terms of performance I have not seen any numbers, but suspect libuv may be the fastest, due to its lightweightedness?

https://ghc.haskell.org/trac/ghc/ticket/8400

Upvotes: 3

Jon Watte
Jon Watte

Reputation: 7198

The real answer, which was indirectly pointed to by Peter Teoh, is based on io_setup() and io_submit(). Specifically, the "aio_" functions indicated by Peter are part of the glibc user-level emulation based on threads, which is not an efficient implementation. The real answer is in:

io_submit(2)
io_setup(2)
io_cancel(2)
io_destroy(2)
io_getevents(2)

Note that the man page, dated 2012-08, says that this implementation has not yet matured to the point where it can replace the glibc user-space emulation:

http://man7.org/linux/man-pages/man7/aio.7.html

this implementation hasn't yet matured to the point where the POSIX AIO implementation can be completely reimplemented using the kernel system calls.

So, according to the latest kernel documentation I can find, Linux does not yet have a mature, kernel-based asynchronous I/O model. And, if I assume that the documented model is actually mature, it still doesn't support partial I/O in the sense of recv() vs read().

Upvotes: 20

J-16 SDiZ
J-16 SDiZ

Reputation: 26910

For network socket i/o, when it is "ready", it don't block. That's what the O_NONBLOCK and "ready" means.

For disk i/o, we have posix aio, linux aio, sendfile and friends.

Upvotes: 0

Related Questions