CODEWITHSUNDEEP
clinuxaio
garen96
garen96

Reputation: 185

How to write data into an offset which is not 512*n bytes using linux native AIO?

I'm writing some app like Bittorrent client to download file from net and write it to local file. I will get partial data and write to the file.

For example, I will download a 1GB file, I will get offset 100, data: 312 bytes, offset 1000000, data: 12345, offset 4000000, data: 888 bytes.

I'm using Linux native AIO(io_setup, io_submit, io_getevents), I found this

When using linux kernel AIO, files are required to be opened in O_DIRECT mode. This introduces further requirements of all read and write operations to have their file offset, memory buffer and size be aligned to 512 bytes.

So how can I write data into some offset which is not 512 aligned?

For example, first I write 4 bytes to a file, so I have to do something like this:

fd = open("a.txt", O_CREAT | O_RDWR | O_DIRECT, 0666);
struct iocb cb;
char data[512] = "asdf";
cb.aio_buf = ALIGN(data, 512);
cb.aio_offset = 512;
cb.aio_nbytes = 512;

Then I would like to append data after asdf:

struct iocb cb2;
char data2[512] = "ghij";
cb2.aio_buf = ALIGN(data2, 512);
cb2.aio_offset = 5;
cb2.aio_nbytes = 512;

It will give error when write

Invalid argument (-22)

So how to do it?

Upvotes: 0

Views: 733

Answers (2)

R.. GitHub STOP HELPING ICE
R.. GitHub STOP HELPING ICE

Reputation: 215201

You don't. The Linux AIO API is not useful, especially not for what you're trying to do. It was added for the sake of Oracle, who wanted to bypass the kernel's filesystems and block device buffer layer for Reasons™. It does not have anything to do with POSIX AIO or other things reasonable people mean when they talk about "AIO".

Upvotes: 1

Ben Voigt
Ben Voigt

Reputation: 283624

You have to do what the driver would do if you weren't using O_DIRECT. That is, read the whole block, update the portion you want, and write it back. Block devices simply don't allow smaller accesses.

Doing it yourself can be more efficient (for example, you can update a number of disconnected sequences in the same block for the cost of one read and write). However, since you aren't letting the driver do the work you also aren't getting any atomicity guarantees across the read-modify-write operation.

Upvotes: 2

Related Questions