CODEWITHSUNDEEP
clinuxio
dragon135
dragon135

Reputation: 1376

How to make sure data integrity after sync/fsync/syncfs to portable device

Based on sync manual page, there is no guarantee the disc will flush its cache after calling sync: "According to the standard specification (e.g., POSIX.1-2001), sync() schedules the writes, but may return before the actual writing is done. However, since version 1.3.20 Linux does actually wait. (This still does not guarantee data integrity: modern disks have large caches.) "

And, in fsync manual, there is no mention about this.

Is there ways to makes sure all writes to disc especially portable device (USB) has been finished after calling sync? I have encountered cases that data and metadata information has not fully written to disc after calling sync/fsync. I am curious how "Safely remove device" in windows/linux knows that all data has been fully written by the device itself.

Upvotes: 1

Views: 1096

Answers (2)

alk
alk

Reputation: 70971

I am curious how "Safely remove device" in windows/linux knows that all data has been fully written by the device itself.

For IXish systems:

Unmount the USB-device's partitions using the umount command or the umount() system call.

Doing 

blockdev --flushbufs

might flush the device's buffer, but does not keep anybody from accessing the devices again and refill buffers.

Also there is this kernel interface in the /proc file system:

/proc/sys/vm/drop_caches

which can be used to flush different buffers:

Verbatim from https://www.kernel.org/doc/Documentation/sysctl/vm.txt

[...]

To free dentries and inodes:

  echo 2 > /proc/sys/vm/drop_caches

[...]

Upvotes: 2

R.. GitHub STOP HELPING ICE
R.. GitHub STOP HELPING ICE

Reputation: 215427

At least in principle, this is a Linux bug. The specification for sync functions is that the data is fully written to permanent storage; leaving it in a hardware cache is not conforming.

I'm not sure what the correct workaround is, but you can probably strace the hwparm utility running with the -F option (I think that's the right one) to see what it's doing (or read the source, but strace is a lot easier).

Upvotes: 1

Related Questions