CODEWITHSUNDEEP
javascriptnode.jsweb-scrapingpuppeteer
Menne
Menne

Reputation: 127

puppeteer execute a js function on the chosen page

This is the analyzed page https://www.diretta.it/.

In this page the content of the following days is loaded dynamically with the js without changing the URL of the site (you can try it at the top right of the table).

Using puppeteer, with the following code

await page.goto ('https://www.diretta.it/');

it loads the contents of today's page. Is there a way to load the page with tomorrow's content? i have to scrape information from the matches of the following days

the function in js executable from terminal for change day is:

> set_calendar_date ('1')

Upvotes: 12

Views: 19263

Answers (1)

Azami
Azami

Reputation: 2161

What you are looking for is the page.evaluate() function. This function lets you run any JS function in the page context.

In simpler terms, running page.evaluate() is akin to opening Dev tools and writing set_calendar_date('1') there directly.

Here is a working snippet, don't hesitate to pass {headless: false} to puppeteer.launch() if you want to see it working with your own eyes.

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://www.diretta.it/');
  await page.evaluate(() => {
    set_calendar_date ('1');
  });
  await page.waitFor(500); //Wait a bit for the website to refresh contents

  //Updated table is now available
})();

Upvotes: 15

Related Questions