How to wait while file is written

Question

I have many processes which write files (any file can be written once). They open, write and close files. Also i have many processes which are read files. File size can various. need such: when some process tries to read file which is writing at this moment, i need to read full content when file is closed after write. I need to lock on write and wait for unlocked on read. Important: if file reads file and can't do that it writes file by itself. 1. Try to read file 2. If file does not exists, write file

So for async mode there can be more than 1 process who want to write file because of can't read it. I need to lock file writing and all readers should wait for this

Answer 1

File locking is an operating-system specific thing.

• https://en.wikipedia.org/wiki/File_locking

Unix-Like systems

Unix-like systems generally support the flock(), fcntl() and lockf() system calls. However apart from lockf advisory locks, it is not part of the Posix standard, so you need to consult operating-system specific documentation.

Documentation for Linux is here:

• http://linux.die.net/man/3/lockf
• http://linux.die.net/man/2/fcntl
• http://linux.die.net/man/2/flock

Note that fcntl() does many things not just locking.

Note also that in most cases locking on unix-like systems is advisory - i.e. a cooperative effort. Both parties must participate and simply ignoring the lock is a possibility. Mandatory locking is possible, but not used in the typical paradigm.

Windows

In windows mandatory file locks (Share mode with CreateFile) and range locks LockFileEx are normal, and advisory locks are not available, though they can be emulated (typically with a one-byte range lock at 0xffffffff or 0xffffffffffffffff - the portion locked does not have to actually exist so this does not imply that the file is that big).

Alternatives

An alternative for your described scenario, is to simply create the file with a different name, then rename it when done.

E.g. if the file is to be called "data-20130719-112258-99823.csv" instead create one called "tmpdata-20130719-112258-99823.csv.tmp" then when it has been fully written, rename it.

Answer 2

The standard way to handle this issue is to write to a temp file name, then rename the file when writing is complete.

Other processes waiting for the file need to watch for the existence of the real file (using a file system watcher, or similar mechanism). When the file "appears", the writing is already complete.