CODEWITHSUNDEEP
javascriptpuppeteer
MichaelBae
MichaelBae

Reputation: 19

Looping inside a page.evaluate in Puppeteer

I have a loop inside a page.evaluate method. The loop iterates a query selector which catches an innerText from multiple instances of a text element in a page.

I am receiving an error Evaluation Failed: Cannot read property of 'innerText'

I tried to loop outside of page.evaluate, but my iteration variable is not accessible from within the page.evaluate function.

// Here's a rough draft of what i'm trying to achieve:

    const scrapeData = [];
    const data = await page.evaluate(() => {

    // Iteration to capture each target text in the page  
    for (var i = 1; i < 9; i++) {

        // Select target text 
        const serpDesc = document
        .querySelector(
          `#rso > div:nth-child(4) > div > div:nth-child(${i}) > div > div > div.s > div > span`
        )
        .innerText.trim();

      // Build an array for the captured text
      scrapeData[i] = serpDesc

      return {
        serpDesc
      };
    };
  });

My goal is to scrape some link descriptions(plain text) from a page into an array. Without the iteration code, everything works fine.

Upvotes: 0

Views: 1965

Answers (1)

Will
Will

Reputation: 131

Try:

const serpDesc = await page.evaluate(
  () => [...document.querySelectorAll(`#rso > div:nth-child(4) > div > div:nth-child(${i}) > div > div > div.s > div > span`)].map(elem => elem.innerText)
);

You will probably need to reconstruct your selector a bit, or maybe wrap serpDesc function in a for of or forEach loop.

You could also try something like

async function elSelector(i) {
            //Where i is the incremented value you pass
            
            await page.evaluate((i) => {
            
                let eval = $('yourSelector').toArray();
                $(eval[i]).innerText
            }, i)

        }
        
for (i=0; i<9; i++) {
  elSelector(i);
  }

Upvotes: 0

Related Questions