CODEWITHSUNDEEP
javascriptnode.jsloopsasynchronousnpm
Rxchard
Rxchard

Reputation: 341

Node.js - Wait for multiple async calls to finish before continuing in code

So basically i have a for loop with an async function in it. Problem is that the program just continues after the loop and i want it to wait until all async functions which were called in the loop are finished before the code continues.

In my code "bar" is a json array with other json arrays in it.

function write(bla) { // gets called one after another

  for(var url in bla) {
    asyncFunctionCall(url); // Executed about 50 times, it has to run parallel
  }
  // Wait for all called functions to finish before next stuff happens and
  // write gets called again.

}

for(var foo in bar) {
  // Here i parse the json array "foo" which is in the json array "bar"
  write(foo[bla]); // bla is an array of multiple urls.
}

The async function call looks something like this:

var request = require('request');

request(url, function (error, response, body) {
  if(typeof response !== 'undefined') {
    if((response.statusCode >= 400 && response.statusCode <= 451)
    || (response.statusCode >= 500 && response.statusCode <= 511))
      return true;
    return false;
  }
  return false;
});

Upvotes: 23

Views: 32983

Answers (2)

T.J. Crowder
T.J. Crowder

Reputation: 1074038

The simplest way here is to use promises, directly or via async/await syntax. In this case, probably directly.

First, you have to make asyncFunctionCall return a promise. It looks like you always return a boolean, so in this case we'll always resolve the promise:

function asyncFunctionCall(url) {
  return new Promise(resolve => {
    request(url, function (error, response, body) {
      if(typeof response !== 'undefined') {
        if((response.statusCode >= 400 && response.statusCode <= 451)
        || (response.statusCode >= 500 && response.statusCode <= 511)) {
          resolve(true);
          return;
        }
      }
      resolve(false);
    });
  });
}

Then, build up an array of your promises, and use Promise.all to wait for all of them to complete. These calls run in parallel:

function write(bla) { // gets called one after another
  const promises = [];
  for(var url in bla) {
    promises.push(asyncFunctionCall(url)); // Executed about 50 times.
  }
  return Promise.all(promises);
}

Then you can build a chain of all of the promises from write so they run in series:

let p = Promise.resolve();
for (const foo in bar) { // <== Notice `const`
  // See "Edit" below
  p = p.then(() => {
    // Here i parse the json object "foo" in the json array "bar"
    // bla is an array of multiple urls.
    return write(foo[bla]));
  });
}

Note that it's important that you use const or let, not var, for foo in that loop, because the then callback closes over it; see this question's answers for why const and let make that work.

Each call to write will only be made when the previous one's work is done.

Then wait for the whole process to complete:

p.then(() => {
  // All done
});

You haven't shown anything using the booleans from write's requests, but they're available (as an array) as the resolution value of the promise from write.

The second part of the process, where we're calling write, can also be written in an async function which may make the logical flow clearer:

async function doTheWrites() {
  for (const foo in bar) {
    // Here i parse the json object "foo" in the json array "bar"
    // bla is an array of multiple urls.
    await write(foo[bla]);
  }
}

Then the overall process:

doTheWrites().then(() => {
  // All done
});

...or if this is also in an async function:

await doTheWrites();

Upvotes: 23

Jonas Wilms
Jonas Wilms

Reputation: 138247

Make the functions async and await the calls:

async function write(foo) {
   for(const url of foo) {
      await asyncFunctionCall(url);
   }  
}

(async function() {
   for(const foo of bar) {
     await  write(foo);
   }
})()

That will execute one request after another. To execute them on parallel use Promise.all:

const write = foo => Promise.all(foo.map(asyncFunctionCall));

Promise.all(bar.map(write))
  .then(() => console.log("all done"));

Upvotes: 4

Related Questions