CODEWITHSUNDEEP
node.js
crawf
crawf

Reputation: 9658

node.js readFile race condition - reading file when not fully written to disk

My node.js app runs a function every second to (recursively) read a directory tree for .json files. These files are uploaded to the server via FTP from clients, and are placed in the folder that the node script is running on.

What I've found (at least what I think is happening), is that node is not waiting for the .json file to be fully written before trying to read it, and as such, is throwing a 'Unexpected end of input' error. It seems as though the filesystem needs a few seconds (milliseconds maybe) to write the file properly. This could also have something to do with the file being written from FTP (overheads possibly, I'm totally guessing here...)

Is there a way that we can wait for the file to be fully written to the filesytem before trying to read it with node?

fs.readFile(file, 'utf8', function(err, data) {
    var json = JSON.parse(data); // throws error
});

Upvotes: 1

Views: 1264

Answers (1)

Robert Peters
Robert Peters

Reputation: 4104

You can check to see if the file is still growing with this:

https://github.com/felixge/node-growing-file

Upvotes: 1

Related Questions