CODEWITHSUNDEEP
javamultithreadingperformancethread-safetythreadpool
Kamlesh Kanazariya
Kamlesh Kanazariya

Reputation: 1259

How to synchronize file wise file writer object?

Currently i am designing my product architect and facing below issue.

I have a Queue in which i am putting object which contain a string and file name in which string should be written.

Suppose i have 10 files say OneFile.txt , twoFile.txt etc. Now i have three object of OneFile.txt , five object of twoFile.txt in Queue.

I have multiple thread which writes to file. Now if i synchronize file writer object then it is block my other threads which not writing to same file.

Suppose Two thread T1 and T2 are there. T1 Thread is writing to OneFile.txt. T2 Thread wants to write to twoFile.txt then T2 thread blocks as T1 is writing to OneFile.txt. T2 should not be block as it wants to write twoFile.txt.

Please suggest what is good architecture. Please provide sample code for that.

Upvotes: 0

Views: 112

Answers (2)

Ivan Senic
Ivan Senic

Reputation: 649

Seams like you could use the AsynchronousFileChannel (considering you are running Java 1.7+). You would open a channel per file that you need to want to write to (see open method).

This type of channel provides the asynchronus write operation where you specify from which position in the file the write should occur - AsynchronousFileChannel#write(java.nio.ByteBuffer, long, A, java.nio.channels.CompletionHandler). This enables you to also do parallel writes to the same text file as well. The only thing you need to do is to take care of the "next write position". You can do that by keeping one AtomicLong per channel and use CompareAndSwap to execute concurrent updates:

Here is the peace of code from the CustomAsyncChannel class from our project:

/**
 * Reserves the writing position in this channel with the given size. This method is thread
 * safe.
 * 
 * @param writeSize
 *            Size of writing that has to be done.
 * @return Returns the position where file should be written.
 */
public long reserveWritingPosition(long writeSize) {
    while (true) {
        long writingPosition = nextWritingPosition.get();
        if (nextWritingPosition.compareAndSet(writingPosition, writingPosition + writeSize)) {
            return writingPosition;
        }
    }
}

The class itself should be a pretty good guide for you. Note that the write can be only partially done, which you can check in the CompletionHandler passed to the write operation.

Upvotes: 0

Enno Shioji
Enno Shioji

Reputation: 26882

It's not obvious whether it's a good idea to write in parallel. E.g. on a hard disk, you'll normally make it slower, on SSDs it can be faster, but it will depend on your situation (size of file, what processing is done before the actual IO etc.). It's also not obvious if you need it to be faster.

So just use a single thread until you've determined it to be the bottleneck.

Upvotes: 1

Related Questions