Incredibly weird puppeteer behaviour

Question

I use this code:

// <-- add event on top of file
process.on("unhandledRejection", (reason, p) => {
        console.error("Unhandled Rejection at: Promise", p, "reason:", reason);
        // browser.close(); // <-- no need to close the browser here
});

const puppeteer = require('puppeteer');
async function getPic() {
    try{ // <-- wrap the whole block in try catch
      const browser = await puppeteer.launch(/*{headless: false}*/);
      const page = await browser.newPage();
      await page.setViewport({width: 1000, height: 500}); // <-- add await here so it sets viewport after it creates the page


      //await page.goto('https://www.google.com');  //Old way of doing. It doesn't work for some reason...

      page.goto('https://www.google.com/').catch(error => console.log("error on page.goto()", error));


      // wait for either of events to trigger
      await Promise.race([
        page.waitForNavigation({waitUntil: 'domcontentloaded'}),
        page.waitForNavigation({waitUntil: 'load'})
      ]);


      await page.screenshot({path: 'pic.png'});
      await browser.close(); // <-- close browser after everything is done
    } catch (error) {
      console.log(error);
    }
}

getPic();

Then the program hangs. After 30 seconds, i get this error:

error on page.goto() Error: Navigation T imeout Exceeded: 30000ms exceeded at Promise.then (C:\...\pupet test\node_modules\puppeteer\lib\NavigatorWatcher.js:71:21) at <anonymous>

But i also get the picture i requested!

1.So how is it that page.goto() fails but it still gets the picture, which mean that page.goto() actually worked!? 2. What can i do to mitigate this weird error?

Answer 1

The program hangs because you called goto without async-await or promises, then you put it in a race for waitForNavigation, this makes the browser confused because all three line of code is mostly doing same thing on the back. It is trying to navigate and wait for it.

Use async await for promises. Do not call async methods synchronous ways. No matter what, this is how you must use it in your example case.

await page.goto('https://www.google.com');

If you want to wait until page load, then the goto function has that covered too. You don't need to use the waitForNavigation after goto.

await page.goto('https://www.google.com', {waitUntil: 'load'}); 

There is also domcontentloaded, networkidle2, networkidle0 for the waitUntil property. You can read more about it in the docs with full explanation.

The reason why screenshot is working properly is because it's getting executed asynchronously but then you are awaiting for the navigation later on.

Here is the code without much complexity and promise race.

try{ // <-- wrap the whole block in try catch
      const browser = await puppeteer.launch(/*{headless: false}*/);
      const page = await browser.newPage();
      await page.setViewport({width: 1000, height: 500}); // <-- add await here so it sets viewport after it creates the page
      await page.goto('https://www.google.com/', {waitUntil: 'load'})
      await page.screenshot({path: 'pic.png'});
      await browser.close(); // <-- close browser after everything is done
    } catch (error) {
      console.log(error);
    }

Here is how it works perfectly on the sandbox.

The puppeteer docs is a good place to start to learn about this.