CODEWITHSUNDEEP
linuxfileposixpersistencedurability
yuri kilochek
yuri kilochek

Reputation: 13589

What are posix/linux/filesystem durability guarantees for ordered file writes?

Suppose I have a (single-threaded) program that (exclusively) opens a large file and does two non-overlapping writes:

int fd = open(path, O_RDWR);
pwrite(fd, data1, size1, offset1);
pwrite(fd, data2, size2, offset2);
close(fd);

Are there any guarantees (by posix, linux or common filesystems like ext4) that, in case of power failure, no part of data2 will end up in permanent storage unless all of data1 also ends up in permanent storage?

Or, to put it another way, that the file (in permanent storage) won't end up in a state where the second write started while the first hadn't completed?

Or do I have to fsync(fd)/fdatasync(fd) between the writes to achieve this?

Upvotes: 1

Views: 290

Answers (1)

Mayank Kumar Chaudhari
Mayank Kumar Chaudhari

Reputation: 18686

There are no guarantees that writes issued to a file are durable (i.e., written to stable storage) or that they maintain a specific ordering without explicit synchronization.

If durability and ordering is important (e.g., for power failure safety), you should:

  1. Write the first chunk (data1).
  2. Call fsync(fd) to ensure it is committed to stable storage.
  3. Write the second chunk (data2).
  4. Optionally, call fsync(fd) again to persist data2.

This should guarantee both the ordering and durability of your writes.

A Reference worth noting - https://puzpuzpuz.dev/the-secret-life-of-fsync

Upvotes: 1

Related Questions