CODEWITHSUNDEEP
springspring-integrationspring-batch
Varun Maurya
Varun Maurya

Reputation: 337

How to use File lock in Spring batch-integration?

In my spring-batch-integration app, file polling invokes the batchjob for eachfile and this application could be running on multiple servers(nodes) but they all are supposed to read a common directory.Now, I wrote a custom locker which takes the lock on file so that any other instance will not be able to process the same file . code as below

public class MyFileLocker extends AbstractFileLockerFilter{

private final ConcurrentMap<File, FileLock> lockCache = new ConcurrentHashMap<File, FileLock>();
private final ConcurrentMap<File, FileChannel> ChannelCache = new ConcurrentHashMap<File, FileChannel>();

@Override
public boolean lock(File fileToLock) {
    FileChannel channel;
    FileLock lock;
    try {
        channel = new RandomAccessFile(fileToLock, "rw").getChannel();
        lock = channel.tryLock();

        if (lock == null || !lock.isValid()) {  
            System.out.println(" Problem in acquiring lock!!" + fileToLock);
            return false;
        }
        lockCache.put(fileToLock, lock);
        ChannelCache.put(fileToLock, channel);

    } catch (FileNotFoundException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

    return true;
}

@Override
public boolean isLockable(File file) {

    return file.canWrite();
}

@Override
public void unlock(File fileToUnlock) {
    FileLock lock = lockCache.get(fileToUnlock);
    try {
    if(lock!=null){
        lock.release();
        ChannelCache.get(fileToUnlock).close();
    }
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

}

}

Now, when i invoke my Spring batch and i try to read that file using flatfileitemreader it gives me

org.springframework.batch.item.file.NonTransientFlatFileException

which i believe is coming beacuse file is locked. I did some googling and found that NIOLocker locks the file in a way that even the current thread can't read it. I found a link where it shows how to read the locked file but they are using buffer. How can I make my file accessible to my FlatfileItemReader.

Please suggest.

Upvotes: 1

Views: 2428

Answers (3)

Varun Maurya
Varun Maurya

Reputation: 337

Solution : I am creating a temporary file with content of the locked file and processing it. Once processing is done i archive that file and remove the locked and temporary both files. Key here is to create a new file with locked file content. code for below is as follows : 


File tmpFile = new File(inputFile.getAbsolutePath() + ".lck");     
FileChannel fileChannel = MyFileLocker.getChannelCache().get(new File(inputFile.getAbsolutePath()));
InputStream inputStream = Channels.newInputStream(fileChannel);
ByteStreams.copy(inputStream, Files.newOutputStreamSupplier(tmpFile));

Here inputFile is my locked file and tmp file is new file with locked file content. I have also created a method in my locker class to unlock and delete

    public void unlockAndDelete(File fileToUnlockandDelete) {
    FileLock lock = lockCache.get(fileToUnlockandDelete);
    String fileName = fileToUnlockandDelete.getName();
    try {
    if(lock!=null){
        lock.release();
        channelCache.get(fileToUnlockandDelete).close();
        //remove from cache
        lockCache.remove(fileToUnlockandDelete);
        channelCache.remove(fileToUnlockandDelete);
        boolean isFiledeleted = fileToUnlockandDelete.delete();
        if(isFiledeleted){
            System.out.println("File deleted successfully" + fileName);
        }else{
            System.out.println("File is not deleted."+fileName);
        }
    }
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

}

Upvotes: 0

Karthik Cherala
Karthik Cherala

Reputation: 322

Based on the link that you shared, looks like you could try the following.

//This is from your link (except the size variable)
FileChannel fileChannel = channelCache.get(lockedFile);
int size = (int) fileChannel.size();
ByteBuffer byteBuffer = ByteBuffer.allocate(size);
fileChannel.read(byteBuffer);

//Additional code that you could try
byte[] bArray = new byte[size];

//Write data to the byte array
byteBuffer.get(bArray);

FlatFileItemReader flatFileItemReader = new FlatFileItemReader();
flatFileItemReader.setResource(new ByteArrayResource(bArray));

//Next you can try reading from your flatFileItemReader as usual
...

Let me know if it doesn't progress your issue.

Upvotes: 1

Artem Bilan
Artem Bilan

Reputation: 121560

Yes, you really can get access to the locked file content only over ByteBuffer:

FileChannel fileChannel = channelCache.get(lockedFile);
ByteBuffer byteBuffer = ByteBuffer.allocate((int) fileChannel.size());
fileChannel.read(byteBuffer);
System.out.println("Read File " + lockedFile.getName() + " with content: " + new String(byteBuffer.array()));

Oh! Yeah. You really pointed to my repo :-).

So, with locker you don't have choice unless copy/paste the file byte[] that way before FlatfileItemReader or just inject some custom BufferedReaderFactory into the same FlatfileItemReader, which converts the locked file to the appropriate BufferedReader:

new BufferedReader(new CharArrayReader(byteBuffer.asCharBuffer().array()));

Upvotes: 2

Related Questions