Single "lock" or separate reader/writer locks for IO operations?

Question

I have a few threads writing and reading different files.

Is it ok to use a single lock {} (the same variable for all protected regions) for all disk operations? So I don't have two threads simultaneously reading and writing to disk to about seeks?

I also heard that I could also use on thread for reads and another for writes, is this always true? why?

Answer 1

Your requirement seems somewhat confusing and morphs. One comment says 'the threads are writing to the same file' and another 'all write to the same collection of files simultaneously'.

There are some choices:

1) Lock up the reads and writes with one lock. this is the simplest method but has the highest probability of contention between the calling threads because the lock is held for the duration of a disk operation.

2) Lock up the reads and writes with one reader/writer lock per file - this is better than (1) in that contentin on different files does not happen. There could still be contention between reads/writes to the same file.

2) Queueing off the reads/writes to one writer thread. This tends to exercise the disk more because it has to swap around between files as it dequeues and executes write requests, but minimizes write contention in the calling threads - they only have to lock a queue for the time taken to push a pointer on. Reading becomes a slow operation because the calling threads would have to wait on a synchro object until their read request is completed. Low contention on writes but high latency on all reads.

3) Like (2), but using a thread per file. This can get expensive memory-wise for several files and only really helps over (2) if the output files are spread over several physical disks. Like (2), low contention and slow reads.

4) Queueing off the writes as threadpool tasks. I'm not sure how to do this exactly - the file context would have to be passed as a parameter and access to it would probably need locking up - this may not work effectively. Like (2), low contention and slow reads.

5) Redesign your app to avoid this requirement entirely?

Answer 2

Using only one lock could slow your application. If a thread is writing a file for a long time, maybe other threads should be allowed to read some other files. Could you be more precise on how which threads access which files?

Answer 3

If each thread reads or writes to a different file, I don't see why you need concurrency.

Usually, there are multiple threads accessing the same file (resource) for reading and writing. In that scenario, when a thread is writing to the file, all the other threads have to wait. This is a classic concurrency problem called "Readers-Writers".

You can find more information here: http://en.wikipedia.org/wiki/Readers-writers_problem

Answer 4

If you are not accessing code of other thread from any thread then one object for synchronization would be enough but it would increase the thread queue waiting for resource. One sync object for each resource or group of resource would be better option