CODEWITHSUNDEEP
javascriptnode.jsweb-scrapingpuppeteer
Mana
Mana

Reputation: 95

Trying to execute a for loop within the page.evaluate() function - Puppeteer

I am currently using Puppeteer to scrape a web page. The web page specifically has div elements of nth-child that I am trying to iterate through. Without a for-loop I can successfully retrieve the innerText of a div.ticker element but if I add a for-loop to retrieve the innerText of multiple div.ticker elements, then the code within for-loop fails to execute and instead jumps straight to line 11. No errors are surfacing upon execution which is making me struggle to find why my for-loop is not executing.

 let scrapeData = await page.evaluate(() => {
    for (let i = 1; i < 21; i++) {
      let element = document.querySelector("#optionflow > div.component-body.ps.ps--theme_default.ps--active-y > div.data-body > div:nth-child(" + i + ") > div.ticker").innerText;
      console.log(element);
    }

    return {
      element
    }
  });
  console.log("Hello world!");
  await browser.close();

  return scrapeData;
}

Upvotes: 0

Views: 1010

Answers (1)

Konrad
Konrad

Reputation: 24691

The element in return statement is not defined. You are declaring the element in the for loop scope and it's destroyed after the loop ends.

If there are multiple elements use array and push elements. But What you should really do is us querySelectorAll without the loop.

Upvotes: 3

Related Questions