CODEWITHSUNDEEP
linuxbash
Mihaimyh
Mihaimyh

Reputation: 1410

Check file is in use over NFS

I have 2 servers which shares a NFS. One of them is writing a file over NFS and the other reads it and sends it over to be processed.

The problem is the second server sometimes reads the file even if it was not fully written by the first server.

I would like to create a script to check if the file is in use, using lsof command, and if it's not then the second server can start processing the file, and if it is to wait for 1 minute for example, then check again.

#! /usr/bin/bash
FILE=$1

function isFileInUse {
lsof $FILE
if [ "$?" -ne "0" ]; then
    exit 0
else
    sleep 60
    isFileInUse
    exit 0  
fi
}

isFileInUse

Do you think the above script can correctly be run on the second server to check if the file is in use by the first server?

Thanks in advance.

Upvotes: 0

Views: 1004

Answers (1)

Bodo
Bodo

Reputation: 9855

I propose using a temporary file with renaming and strict rules who creates and deletes the file. Something like these pseudo shell scripts:

Sender

while condition
do
    if [ ! -f transferfile ]
    then
        create_data > transferfile.tmp
        # atomic creation by renaming
        mv transferfile.tmp transferfile
    else
        sleep 10
    fi
done

Receiver

while condition
do
    if [ -f transferfile ]
    then
        use_data < transferfile
        # atomic removal
        rm -f transferfile
    else
        sleep 10
    fi
done

Possible problems

Error handling when one of the servers doesn't do its work in a reasonable time.

Edit: Possible performance improvement

With the algorithms above the sender starts writing the temporary file when the file for transfer to the receiver has been removed by the receiver, and the receiver removes the file when it finished its processing. That means the sender is blocked after preparing a transfer file as long as the receiver is processing the data.

Depending on how the sender is implemented, the sender could start writing to the temporary file immediately after renaming the temporary file to the fransfer file. With this modification the sender is not blocked when the previous transfer file has not yet been removed.

Similar for the receiver: When the receiver is ready to process new data, it can rename the transfer file to an input file name and process the input file. This allows the sender to create the next transfer file earlier.

Sender (improved)

while condition
do
    if [ ! -f transferfile ] && [ -f transferfile.tmp ]
    then
        # atomic creation by renaming
        mv transferfile.tmp transferfile
    fi
    create_some_data >> transferfile.tmp
done

Receiver (improved)

while condition
do
    if [ -f transferfile ]
    then
        # atomic removal by renaming
        mv -f transferfile inputfile

        # processing from renamed file
        use_data < inputfile
        rm -f inputfile
    else
        sleep 10
    fi
done

Upvotes: 1

Related Questions