CODEWITHSUNDEEP
javascriptnode.jspuppeteer
Bill Mayheptad Ritchie
Bill Mayheptad Ritchie

Reputation: 63

How to get the content of a div tag when scraping with puppeteer and NodeJs

I heard of this library called puppeteer and is usefulness in scraping web pages. so i decided to scrape a gaming site content so I can store it data and go through it later.

But after i copied the XPATH of the div tag I want puppeteer to scrape it content, its returning Empty string Please what am I doing wrong.

This is the url am trying to scrape here

i want to scrape the div tag where the result of the 6 different color ball are being displayed. so i can get the number of those colors every 45 seconds.

const puppeteer = require("puppeteer");

async function scrapeData(url){
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto(url);

const [dataReceived] = await page.$x('/html/body/div[1]/div/div/div/footer/div[2]/div[1]/div/div[1]/div[2]/div/div');
const elContent = await dataReceived.getProperty('textContent');
const elValue = await elContent.jsonValue();
console.log({elValue});
//console.log(elContent);
//console.log(dataReceived)
browser.close();
}
scrapeData("https://logigames.bet9ja.com/Games/Launcher?gameId=11000&provider=0&sid=&pff=1&skin=201");
console.log("just testing");

here is the image of the returned empty string in vscode terminal

enter image description here

Upvotes: 1

Views: 1221

Answers (1)

Zac Anger
Zac Anger

Reputation: 7747

Rather than using page.$x here, you could use a simpler selector, which would be less brittle. Try page.$('.ball-value'), or possibly page.waitForSelector('.ball-value') to deal with transition times. Testing on that page using a simpler selector seems to work. If you want to get all the ball values rather than just the first one, there's page.$$ (which is the same as document.querySelectorAll, so it would return an array of elements).

Upvotes: 1

Related Questions