CODEWITHSUNDEEP
node.jsweb-scrapingpuppeteer
f1rstsurf
f1rstsurf

Reputation: 647

Puppeteer : use second match with page.evaluate

i'm using puppeteer to retrieve datas online, and facing an issue.

Two functions have the same name and return serialized object, the first one returns an empty object, but the second one does contains the datas i'm targeting.

My question is, how can I proceed to select the second occurence of the function instead of the first one, which return an empty object.

Thanks.

My code :

const puppeteer = require('puppeteer');
const cheerio = require('cheerio');

const Variants = require('./variants.js');
const Feedback = require('./feedback.js');

async function Scraper(productId, feedbackLimit) {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  /** Scrape page for details */
  await page.goto(`${productId}`);
  const data = (await page.evaluate()).match(/window.runParams = {"result/)

  const data = data.items

 await page.close();
 await browser.close();


  console.log(data);
  return data;
}

module.exports = Scraper;

Website source code :

window.runParams = {};
window.runParams = {"resultCount":19449,"seoFeaturedSnippet":};

Upvotes: 1

Views: 745

Answers (1)

Or Assayag
Or Assayag

Reputation: 6336

Please try this, it should work.

const data = await page.content();
const regexp = /window.runParams/g;
const matches = string.matchAll(regexp);
    
for (const match of matches) {
  console.log(match);
  console.log(match.index)
}

Upvotes: 1

Related Questions