CODEWITHSUNDEEP
python-2.7lockingshared-file
Daniel
Daniel

Reputation: 1574

is there a way to synchronize write to a file among different processes (not threads)

My project requires being run on several different physical machines, which have shared file system among them. One problem arising out of this is how to synchronize write to a common single file. With threads, that can be easily achieved with locks, however my program consists of processes distributed on different machines, which I have no idea how to synchronize. In theory, any way to check whether a file is being opened right now or any lock-like solutions will do, but I just cannot crack out this by myself. A python way would be particularly appreciated.

Upvotes: 4

Views: 2156

Answers (2)

Chand Priyankara
Chand Priyankara

Reputation: 6782

There are file locking mechanism to multiple process, even written from multiple languages. Operating System specific locking mechanisms are used for this. Java JNI, C++, etc languages have implemented these locking mechanisms to synchronize file access within multiple OS Processes[within LTs-Lightweight Threads].

Look in to your Language specific Native file synchronization mechanisms for this.

Following is a Java based Sample:

FileInputStream in = new FileInputStream(file);
try {
    java.nio.channels.FileLock lock = in.getChannel().lock();
    try {
        Reader reader = new InputStreamReader(in, charset);
        ...
    } finally {
        lock.release();
    }
} finally {
    in.close();
}

This locking should be OS independent [work in Unix like systems, Windows, etc].

For this kind to scenarios, I suggest to use Double Locking for better access controlling.

Upvotes: 0

Arash Sharif
Arash Sharif

Reputation: 592

Just a thought...

Couldn't you put a 'lock' file in the same directory as the file your trying to write to? In your distributed processes check for this lock file. If it exists sleep for x amount and try again. Likewise, when the process that currently has the file open finishes the process deletes the lock file?

So if you have in the simple case 2 processes called A and B:

Process A checks for lock file and if it doesn't exist it creates the lock file and does what it needs to with the file. After it's done it deletes this lock file.

If process A detects the lock file then that means process B has the file, so sleep and try again later....rinse repeat.

Upvotes: 0

Related Questions