CODEWITHSUNDEEP
javascriptnode.jsweb-scrapingpuppeteer
Learner
Learner

Reputation: 1

Failing in WebScraping products names PUPPETEER JS

So this is my code right now

My goal is to WebScrape and print in console the first 10 products names in https://www.nike.com.br/Snkrs#estoque

However, it's returning the same name 10 times

"Kybrid S2Pineapple Comprar

Kybrid S2Pineapple Comprar

...

Kybrid S2Pineapple Comprar"

    const puppeteer = require('puppeteer');
    const { stringify } = require('querystring');
    
    async function Teste(){
        const browser = await puppeteer.launch({
            headless: false
        })
        page = await browser.newPage();
        
        await page.goto('https://www.nike.com.br/Snkrs#estoque');
    
        await page.waitForSelector('#DadosPaginacaoEstoque');
    
        var s = 10
        var i = 1
        
        while (i != s){
            w = await page.evaluate('document.querySelector("#DadosPaginacaoEstoque > div:nth-child(1) > div:nth-child("+String(i)+") > div > div.produto__detalhe").innerText');
            console.log(w);
            i = i + 1;
        }
        console.log(s)
    }
    Teste();

How do I fix this?

Upvotes: 0

Views: 146

Answers (2)

PhantomSpooks
PhantomSpooks

Reputation: 3570

Ah I think the page.$$eval function will get you where you need to be.

The reason it’s printing the same value over and over is because your query is returning the same value over and over. You should use page.$$eval to use queryselectorall and abandon your while loop altogether so it'd be something like this

 const puppeteer = require('puppeteer');
 const { stringify } = require('querystring');   
 
async function Teste(){
   const browser = await puppeteer.launch({
     headless: false
   })
   page = await browser.newPage();  
   await page.goto('https://www.nike.com.br/Snkrs#estoque');   
   await page.waitForSelector('#DadosPaginacaoEstoque');
   // from what I've seen this works
   let selector = "#DadosPaginacaoEstoque div.produto__detalhe"
   let products = await page.$$eval(selector, lists=>lists.map(items=>items.innerText))
   console.log(products.splice(0,10))
 }
    Teste();

Upvotes: 1

vsemozhebuty
vsemozhebuty

Reputation: 13822

It seems you just have a small typo with quotes. Try this:

w = await page.evaluate('document.querySelector("#DadosPaginacaoEstoque > div:nth-child(1) > div:nth-child(' + String(i) + ') > div > div.produto__detalhe").innerText');

Upvotes: 0

Related Questions