CODEWITHSUNDEEP
javascriptnode.jsexpresspromise
Jamie Phan
Jamie Phan

Reputation: 876

How to wait all Promises to resolve in ExpressJS?

Background:

I have a simple REST API in ExpressJS that allows phasing multiple pages together. The page numbers are dynamic.

Issue:

Because of performance constraints, I would like to implement async promises when fetching multiple web pages, wait for all of them finished download, phrase them with my desired format, and return back to the output.

After I researched all the stuff about promises and async online (since I'm still new to this async topic), Most of them told me to use Promise.all, but I just can't get it to work somehow.

Current output when navigating to GET /xxx

{
    username: xxx
    parsedHTML: []
}

Targeted output:

{
    username: xxx
    parsedHTML: [
        "BUNCH OF ANALYSED HTML",
        "BUNCH OF ANALYSED HTML",
        "BUNCH OF ANALYSED HTML",
        ...
    ]
}

Code

const express = require("express");
const http = require("http");
const fetch = require('node-fetch');

const app = express();

app.get("/:username", (req, res)=>{
    const username = req.params.username;
    let page = 3; //Will be obtained dynamically. But for now, I'll put a constant here
    res.json({
        username: username,
        parsedHTML: getParsedHTML(username, page),
    });
    console.log("Page sent")
})

function getParsedHTML(username, page) {

    let promises = [];
    let analyses = [];

    for (var i = 1; i < (page + 1); i++) {
        promises.push(fetch(`https://example.com/profile/${username}/?page=${i}`)
                .then((c) => c.text()));
        // console.log(`Added promise`)
    }
    Promise.all(promises).then(()=>{
        for (let i = 0; i < promises.length; i++) {
            let promise = promises[i];
           analyses.push(analyse(promise));

        }
    })
    return analyses;
}

function analyse(html){
    // Some synchronous analyse stuff here
    // Right now it do nothing
    return html;
}

app.listen(3000, () => console.log('API listening on port ' + 3000 + '!'))

Any helps would be appreciated. Thanks a lot.

Upvotes: 1

Views: 2336

Answers (1)

CertainPerformance
CertainPerformance

Reputation: 370619

You're calling Promise.all on the promises correctly, but the getParsedHTML function isn't waiting for that Promise.all call to resolve before returning. So, your res.json is running immediately, synchronously, and the analyses that are returned is an empty array.

Return the Promise.all call instead, and make sure to analyze the responses (from the Promise.all call) rather than the Promises:

return Promise.all(promises).then((responses)=>{
  for (let i = 0; i < responses.length; i++) {
    let response = responses[i];
    analyses.push(analyse(response));
  }
}).then(() => analyses);

But you can significantly clean up your code by mapping the resulting array of responses:

function getParsedHTML(username, page) {
  let promises = [];
  for (var i = 1; i < (page + 1); i++) {
    promises.push(fetch(`https://example.com/profile/${username}/?page=${i}`)
                  .then((c) => c.text()));
  }
  return Promise.all(promises)
    .then((responses) => responses.map(analyse));
}

Also make sure for to wait for getParsedHTML to wait for the returned Promises to resolve before sending res.json:

app.get("/:username", (req, res)=>{
  const username = req.params.username;
  let page = 3; //Will be obtained dynamically. But for now, I'll put a constant here
  getParsedHTML(username, page)
    .then((parsedHTML) => {
      res.json({
        username,
        parsedHTML
      });
      console.log("Page sent")
    });
})

Upvotes: 3

Related Questions