CODEWITHSUNDEEP
node.jsfor-loopasynchronous
hamster121
hamster121

Reputation: 388

How to wait for for loop with async requests to finish in node.js?

I want to make multiple requests in node.js to get a couple of external API responses, to combine them into one array. I'm using a for loop to achieve this. Here's my code:

res.setHeader('Content-Type', 'application/json');
const sub = req.query.days_subtract;
const enddate = req.query.end_date; 

var array = [];

for (var i = 0; i < sub; i++) {
  request("https://api.nasa.gov/planetary/apod?date=" + subtractDate(enddate, i) + "&api_key=DEMO_KEY", function(error, response, body) {
    array.push(body); 
    // console.log(body);
  });
}
res.send(array);

but this piece of code returns [] all the time. I know it's because the for loop only starts these async requests, but it doesn't wait for them to finish. I tried to use async/await but that didn't work either. So how to wait for this loop to finish getting requests and to finish pushing them to the array so it can be shown to users?

Upvotes: 16

Views: 29351

Answers (4)

Ayush Gupta
Ayush Gupta

Reputation: 9285

For your use case, using await with Promise.all is probably the most performant way to go about it. Your code should look something like this:

res.setHeader('Content-Type', 'application/json');
const sub = req.query.days_subtract;
const enddate = req.query.end_date;

var promiseArray = [];

for (var i = 0; i < sub; i++) {
    promiseArray.push(new Promise((resolve, reject) => {
        request("https://api.nasa.gov/planetary/apod?date=" + subtractDate(enddate, i) + "&api_key=DEMO_KEY", function(error, response, body) {
        if (error) reject(error);
        else resolve(body)
    })
  }))
}
res.send(await Promise.all(promiseArray));

Upvotes: 28

Kon
Kon

Reputation: 4099

Use something like request-promise. If you await each request in the for loop, then all requests are done in series. If each request is independent and doesn't need to be done one after the other, this is inefficient. The best way to handle something like this is to do them all in parallel and then wait for them all to complete. This is done by creating an array of request promises and then using await Promise.all(promiseArray) to wait for all promises to resolve.

var promises = [];

for (var i = 0; i < sub; i++) {
  const promise = request("https://api.nasa.gov/planetary/apod?date=" + subtractDate(enddate, i) + "&api_key=DEMO_KEY");
  promises.push(promise);
}

const array = await Promise.all(promises);

Upvotes: 9

Aymeric Bouzy aybbyk
Aymeric Bouzy aybbyk

Reputation: 2241

As others have said, you should probably use Promise for your asynchronous code, and the async / await syntax is very legible.

However, you seem to prefer using callbacks. In that case, you could write your code this way : 

res.setHeader('Content-Type', 'application/json');
const sub = req.query.days_subtract;
const enddate = req.query.end_date; 

var array = [];
var resultsCount = 0;

for (var i = 0; i < sub; i++) {
  request("https://api.nasa.gov/planetary/apod?date=" + subtractDate(enddate, i) + "&api_key=DEMO_KEY", function(error, response, body) {
    if (error) { res.status(500) }
    array[i] = body
    // console.log(body);
    resultCount++;
    if (resultCount === sub) {
      res.send(array);
    }
  });
}

Basically, the idea is to call the res.send method only once all requests have returned their result.

Upvotes: 2

Evert
Evert

Reputation: 99515

To use async/await, you need to use a HTTP library that returns promises instead of using callbacks.

It's well worth using a library that can do this. A good option for a different library is node-fetch, which effectively implements the "Fetch API" as implemented by browsers.

You can then simply await fetch(url).

async/await works so well and is becoming so common-place, I can highly recommend switching to frameworks and libraries that treat it as a first-class citizen instead of an afterthought. This includes Express, which is also a callback-based library, instead of a Promise-based one. But that's slightly off-topic.

Upvotes: 2

Related Questions