CODEWITHSUNDEEP
javascriptnode.jspromise
siddharth lakhara
siddharth lakhara

Reputation: 307

Handle dynamic number of promises in Node Js

I'm fetching product's data from a bunch of URLs. The result of these promises (along with product details) may contain a key called nextPage, which has a URL as value. I need to fetch data from that URL too. How can I wait for all the original set of promises + dynamic promises attached based on nextPage key ?

Currently I'm trying the following:

const allUrls = ['url1', 'url2', 'url3'];
const promiseArray = [];

const getData = (url) => {
  const p = axios.get(url).then((data) => {
    if (data.nextPage) {
      // Push this into promises so that the data is not lost
      promiseArray.push(Promise.resolve(data));
      
      // There are more pages to fetch data from
      return getData(data.nextPage);
    }
    // No further pages, so return the result
    return data;
  });
  return p;
}

allUrls.forEach((url) => {
  const p = getData(url);
  promiseArray.push(p);
});

Promise.all(promiseArray).then((data) => {
// Error: data only has the value of allUrls, not dynamically added data
})

As I have put a comment, when the Promise.all resolves, I only get the data of original URLs put in allUrls, not the dynamically added promises.

Upvotes: 0

Views: 207

Answers (1)

eol
eol

Reputation: 24565

The problem is that by the time Promise.all is executed, promiseArray will only contain the promises for the product's data, not the details. You'd need to await each product data promise, and then push the resulting promise to the array (note that I'm using async/await as it allows for cleaner code):

for (const url of allUrls) {
   const p = await getData(url);
   promiseArray.push(p);
}
    
const result = await Promise.all(promiseArray);
console.log(result);

Upvotes: 1

Related Questions