Asynchronous control flow - returning true after a for loop is completely done

Question

I have the following method in Node.js:

var foo = function(items){
    for (var i=0; i<items.length; i++){
        var item = items[i];
        bar(item);
    }
    return true;
}

I want foo to return true after all of the bar's have finished updating. How do I do that?

EDIT: I am looking for a way to do this without any external libraries if possible.

Answer 1

The definition of the for loop being "completely done" depends on whether or not bar is synchronous or asynchronous. If bar is synchronous--perhaps it is doing some long-running computation--then your code already returns when the loop is "completely done".

But based on the part of your question that says

after all of the bar's have finished updating

it seems to be a reasonable assumption that bar is asynchronous--perhaps it is making a network call. But currently it is missing any mechanism to report when it's done. Therefore, the first job is to redefine bar so it has an asynchronous interface. There are two basic kinds of asynchronous interfaces. The older, classical node approach is callbacks. A new approach is promises.

In either case, therefore, you need to start off by redefining bar in one way or another. You cannot use an interface asynchronously unless it is designed to be used that way.

After we have given bar an asynchronous interface, we can then use that from within foo to correctly report when the bars are "completely done".

Callbacks

In the callback case, we change the calling sequence for bar to bar(item, callback), giving it a way to report back when it is done. We will also need a callback on the foo routine, to be invoked when all the bar calls finish. Following your preference to not use libraries, the logic for this could look something like this:

function foo(items, callback) {
  var count = items.length;
  var results = [];

  items.forEach(function(item, idx) {        
    bar(item, function(err, data) {
      if (err) callback(err);
      results[idx] = data;
      if (!count--) callback(null, results);
    });
  });
}

This loops over the items, calling bar for each one. When each bar finishes, we place its result in a results array, and check if this is the final bar, in which case we call the top-level callback.

This would be used as

foo(items, function(err, data) {
  if (err) console.log("Error is", err);
  else console.log("All succeeded, with resulting array of ", data);
});

The above is essentially equivalent to using async.each, as suggested in another answer, as follows:

function foo(items, callback) {
  async.each(items, bar, callback);
}

Waiting for each bar to finish

If you want to wait for each bar to finish before proceeding to the next one, with async it's pretty easy with eachSeries:

function foo(items, callback) {
  async.eachSeries(items, bar, callback);
}

Non-library code would be a bit more complicated:

function foo(items, callback) {
  var results = [];

  function next(i) {
    if (i >= items.length) return callback(results));
    bar(items[i], function barCallback(err, data) {
      if (err) return callback(err);
      results[i] = data;
      next(++i);
    });
  }

  next(0);
}

Here, the callback reporting that each bar is completed (barCallback) calls the next routine with incremented i to kick off the call to the next bar.

Promises

However, you are likely to be better off using promises. In this case, we will design bar to return a promise, instead of invoking a callback. Assuming bar calls some database routine MYDB, the new version of bar is likely to look like this:

function bar(item) {
  return MYDB.findRecordPromise(item.param);
}

If MYDB only provides old-fashioned callback-based interfaces, then the basic approach is

function bar(item) {
  return new Promise(function(resolve, reject) {
    MYDB.findRecord(item.param, function(err, data) {
      if (err) reject(err);
      else resolve(data);
    });
  });
}

Now that we have a promise-based version of bar, we will define foo to return a promise as well, instead of taking an extra callback parameter. Then it's very simple:

function foo(items) {
  return Promise.all(items.map(bar));
}

Promise.all takes an array of promises (which we create from items by mapping over bar), and fulfills when all of them fulfill, with a value of an array of fulfilled values, and rejects when any of them reject.

This is used as

foo(items) . then(
  function(data) { console.log("All succeeded, with resulting array of ", data); },
  function(err)  { console.log("Error is", err); }
);

Waiting for each bar to finish

If, with the promises approach, you want to wait for each bar to finish before the next one is kicked off, then the basic approach is:

function foo(items) {
  return items.reduce(function(promise, item) {
    return promise.then(function() { return bar(item); });
  }, Promise.resolve());
}

This starts off with an "empty" (pre-resolved) promise, then each time through the loop, adds a then to the chain to execute the next bar when the preceding one is finished.

If you want to get fancy and use advanced async functions, with the new await keyword:

async function foo(items) {
  for (var i = 0; i < items.length; i++) {
    await bar(items[i]);
  }
}

Aside

As an aside, the presumption in another answer that you might be looking for an asynchronous interface to a routine (foo) which loops over some calls to bar which are either synchronous, or may be asynchronous but can only be kicked off with no way to know when they completed, seems odd. Consider this logic:

var foo = function(items, fooCallback){
    for (var i=0; i<items.length; i++){
        var item = items[i];
        bar(item);
    }
    fooCallback(true);
}

This does nothing different from the original code, except that instead of returning true, it calls the callback with true. It does not ensure that the bars are actually completed, since, to repeat myself, without giving bar a new asynchronous interface, we have no way to know that they are completed.

The code shown using async.each suffers from a similar flaw:

var async = require('async');

var foo = function(items){
  async.each(items, function(item, callback){
    var _item = item;      // USE OF _item SEEMS UNNECESSARY
    bar(_item);
    callback();            // BAR IS NOT COMPLETED HERE
  },
  function(err){
    return true;           // THIS RETURN VALUE GOES NOWHERE
  });
}

This will merely kick off all the bars, but then immediately call the callback before they are finished. The entire async.each part will finish more or less instantaneously, immediately after which the final function(err) will be invoked; the value of true it returns will simply go into outer space. If you wanted to write it this way, it should be

var foo = function(items, fooCallback){
  async.each(items, function(item, callback){
    bar(item);
    callback();
  },
  function(err){
    fooCallback(true);           // CHANGE THIS
  }
  });
}

but even in this case fooCallback will be called pretty much instantaneously, regardless of whether the bars are finished or whether they returned errors.

Answer 2

There are many options to solve what you want, you just need to define the nature of your funcions foo and bar. Here are some options:

Your original code doesn't need it:

Your foo and bar functions doesn't have any callback in their parameters so they are not asynchronous.

Asuming foo function is asynchronous and bar function is synchronous:

You should rewrite your original code using callbacks, something like this:

var foo = function(items, fooCallback){
    for (var i=0; i<items.length; i++){
        var item = items[i];
        bar(item);
    }
    fooCallback(true);
}

Using a Async:

You should install it at the beggining:

npm install async

You use async, like this.

var async = require('async');

var foo = function(items){
  async.each(items, function(item, callback){
    var _item = item;
    bar(_item);
    callback();
  },
  function(err){
    return true;
  }
  });
}