CODEWITHSUNDEEP
cmultithreadingunix
lazyconfabulator
lazyconfabulator

Reputation: 467

Thread safe multi-file writing

I have a daemon that accepts socket connections and reads or writes a dynamic set of files, depending on the nature of the connection. Because my daemon is multithreaded, the possibility exists that the same file may be written to by more than one thread. Because my list of files is dynamic and not fixed, I'm not sure how to keep one thread from bumping into the other. For performance reasons, I want threads to be writing to different files at the same time, just not the same file at the same time.

Other questions have suggested using mutexes, but I'm not entirely clear how a mutex would help in this scenario - the list of files being dynamic and only known to the thread.

Would it be appropriate to use file locking in this case? If so, how would one implement file locking in a thread-safe way?

Upvotes: 3

Views: 1753

Answers (3)

Johan Kotlinski
Johan Kotlinski

Reputation: 25729

flock will work OK. It doesn't lock file descriptors, it locks the actual file.

A file that has been exclusively flock'ed can't be exclusively locked again by another process or thread. That would defeat the entire purpose of locks.

One note is that these locks are advisory. A process that doesn't use flock can happily overwrite the file, even if another process has exclusive-flock'ed it.

Upvotes: 2

wibarr
wibarr

Reputation: 276

I can't say this would be the "optimum" solution, but I'd propose something like this:

Maintain a linked list of a struct that contains two things:

  1. The filename
  2. A condition wait variable associated with the file.

Flow A. When the daemon receives a request, mutex lock the list and check to see whether the filename is in the list or not. If it is not, add a new entry to the linked list with a new condition wait variable for other threads to use. Release the mutex lock. Perform the file operation. Once complete, lock the linked list and remove the struct entry for that file, then signal the other threads via the wait object.

Flow B. If a request comes in for the same file, it'll lock the list and look for the filename contained in the list. If it is in the list, grab the wait variable and wait on it. When the thread is signaled, grab a lock on the list and see if the file is in the list (It's possible another thread picked up the lock on the filename before you). If not, follow Flow A. If so, grab the wait variable in the new struct and wait again until signaled, then follow the above steps again.

Upvotes: 1

Matthew
Matthew

Reputation: 10444

I would use an event broker pattern. Each socketing thread fires an event (have args of the file(s) ) then the event is handled by a central file broker with a shared collection of files currently being written.

If the file cannot be written to, decide what you want to do... otherwise report a success.

Multiple listeners, one central file-lock collection, multiple writers.

Upvotes: 1

Related Questions