CODEWITHSUNDEEP
node.jsasynchronous
NicholasJohn16
NicholasJohn16

Reputation: 2409

Async Recursive Readdir in Node

I'm trying to scan a large directory of files and sub-directories with a recursive function that I've written. It's very similar to the parallel loop given in this answer. 

var fs = require('fs');
var walk = function(dir, done) {
  var results = [];
  fs.readdir(dir, function(err, list) {
    if (err) return done(err);
    var pending = list.length;
    if (!pending) return done(null, results);
    list.forEach(function(file) {
      file = dir + '/' + file;
      fs.stat(file, function(err, stat) {
        if (stat && stat.isDirectory()) {
          walk(file, function(err, res) {
            results = results.concat(res);
            if (!--pending) done(null, results);
          });
        } else {
          results.push(file);
          if (!--pending) done(null, results);
        }
      });
    });
  });
};

This issue is that it isn't really asynchronous. The whole thing processes and returns a giant array of files. Is there a way to recursively scan a directory asynchronously?

Something more like:

walk(path,function(files) {
  // do stuff with each file as its found
});

EDIT: once I start getting the files, I plan to access them and use the async module to process them and prevent using up file descriptors. So something like this:

walk(path,function(files) {
    async.each(files,function() {
        // do more stuff
    }
});

Will that work okay with an asynchronous scan?

Upvotes: 2

Views: 1401

Answers (1)

Naeem Shaikh
Naeem Shaikh

Reputation: 15725

yes HeadCode already explained it in a comment above. You can use eventEmitter to do this kind of Asynchronous recursive stuff. For your code you can put the walk function as an event callback.

var EventEmitter = require("events").EventEmitter;

var ee = new EventEmitter();
ee.on("walk", YourWalkFunction);

ee.emit("walk",YourParams);

Upvotes: 1

Related Questions