CODEWITHSUNDEEP
node.jsreadfilefs
micnic
micnic

Reputation: 11265

Asynchronously reading and caching multiple files in nodejs

I have an array which keeps URL of several files. For example:

var files = ['1.html', '2.html', '3.html'];

I need to read them asynchronously and save them in an object named cache (cache = {}). To do this I used the code:

for(var i = 0; i < files.length; i++){
    require('fs').readFile(files[i], 'utf8', function (error,data) {
        cache[files[i]]=data;
    });
}

In the end I have the result:

cache = { undefined : 'File 3 content' }

I do understand that the "readFile" acts after the loop is ended and it looses it's scope. Is there a way to fix this or another method to read files from an array and cache them?

Upvotes: 5

Views: 20008

Answers (2)

KatieK
KatieK

Reputation: 13863

The existing answer didn't work for me. I did find an NPM package which did the job: https://www.npmjs.com/package/read-multiple-files. After npm install read-multiple-files at the command line, here's the code I used:

var files = ['1.html', '2.html', '3.html'];

console.log("\n");

readMultipleFiles(files, 'utf8', function(err, inputFiles) {
  if(err) {
    console.log("Read Error: " + err);
  }

  fileOne = inputFiles[0];
  fileTwo = inputFiles[1];
  ...

  console.log(fileOne);
  console.log(fileTwo);

});

Upvotes: 0

Linus Thiel
Linus Thiel

Reputation: 39261

When your callback to readFile executes, the for loop will already have finished. So i will be files.length and files[i] will be undefined. To mitigate this, you need to wrap the variables in a closure. The simplest way to do this is to create a function which does your readFile call, and call that in the loop:

function read(file) {
    require('fs').readFile(file, 'utf8', function (error,data) {
        cache[file]=data;
    });
}

for(var i = 0; i < files.length; i++){
    read(files[i]);
}

For even better execution control, you might want to look into async:

function readAsync(file, callback) {
    fs.readFile(file, 'utf8', callback);
}

async.map(files, readAsync, function(err, results) {
    // results = ['file 1 content', 'file 2 content', ...]
});

Edit: Made use of helper function for async example.

Upvotes: 19

Related Questions