CODEWITHSUNDEEP
javascriptnode.jspuppeteerdocument
Vipin Verma
Vipin Verma

Reputation: 5735

Puppeteer: page.evaluate not working after waitForNavigation, debug not working inside page.evaluate for document

I am able to navigate to a page using puppeteer but afterwards the page.evaluate is not returning any response. Further, I am unable to debug inside the page.evaluate either. I run the script in debug mode (node debug filename.js), use sb(15) to set the breakpoint on line 15, press c to continue, wait for the page to load, then enter 'repl'. Now when I try to debug it says document is not defined. How do I solve these two issues?

const puppeteer = require('puppeteer');

(async function scrape() {
  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();
  page.setDefaultNavigationTimeout(90000);
  const url = "https://excise.wb.gov.in/CHMS/Public/Page/CHMS_Public_Hospital_Bed_Availability.aspx";
  await page.goto(url, {waitUntil: 'networkidle2', timeout: 0});
  await page.waitForSelector('#ctl00_ContentPlaceHolder1_ddl_District');
  await page.select('#ctl00_ContentPlaceHolder1_ddl_District', '020');
  await page.waitForNavigation();
  let beds = await page.evaluate(() => {
    let dataRows = document.body.querySelectorAll("tbody tr");
    console.log("Num entires == " + dataRows.length);
  });
  await browser.close();
})();

Upvotes: 1

Views: 1613

Answers (1)

blex
blex

Reputation: 25648

Selecting a city does not cause the URL to change, which is what page.waitForNavigation() waits for.

This resolves when the page navigates to a new URL or reloads.

It never happens, so your code does not continue.

You might be looking for page.waitForSelector() instead:

  // ...
  await page.select('#ctl00_ContentPlaceHolder1_ddl_District', '020');
  await page.waitForSelector('tbody tr');
  let beds = await page.evaluate(() => {
    let dataRows = document.body.querySelectorAll('tbody tr');
    return [...dataRows].map(row => row.querySelector('h5').textContent);
  });
  console.log(beds);
  await browser.close();

Upvotes: 2

Related Questions