CODEWITHSUNDEEP
node.jspuppeteer
Andrew Kloos
Andrew Kloos

Reputation: 4608

Puppeteer on Raspberry PI on ubuntu sometime gives "Evaluation failed: TypeError: Cannot read property 'textContent' of null"

This code runs correctly locally every time. However, when I deploy to the server (ubuntu on Raspberry Pi using chromium-browser) I sometimes get errors around 3/10 attempts. This code works best...

await page.goto('http://mywebsite.com')
const element = await page.$('div[class="user-tags"]')
const value = await page.evaluate(el => el.textContent, element)

but sometimes returns... "Error Getting Experience Level Error: Evaluation failed: TypeError: Cannot read property 'textContent' of null"

So I looked around for solutions and tried this but it fails every time (both code blocks run locally fine)...

 await page.goto('http://mywebsite.com')
 await page.waitForSelector('div[class="user-tags"]')
 const element = await page.$('div[class="user-tags"]')
 const value = await page.evaluate(el => el.textContent, element)

Which throws " Error Getting Experience Level TimeoutError: waiting for selector "div[class="user-tags"]" failed: timeout 30000ms exceeded 9/10/2020 @ 06:02:35"

Thanks for any suggestions!

Upvotes: 1

Views: 352

Answers (1)

Vaviloff
Vaviloff

Reputation: 16856

The difference between first and second code snippets

In the second code sample you instruct puppeteer to wait until div[class="user-tags"] exists on the target page:

await page.waitForSelector('div[class="user-tags"]')

That is the correct way of getting data from an element - first make sure it is available, then query it.

Timeout error happens because the given element is not found within 30 seconds (it is the default timeout).

Ways to solve this

First you need to figure out why the element is not found by the puppeteer.

  • Maybe div.user-tags is not supposed to be present on every page?
  • Maybe Raspberry Pi is not powerful enough to load and process the target page in 30 seconds? - It is possible to increase the timeout.
  • You can also wait until the page is completely loaded, this way puppeteer will make sure all of the resources are loaded before going on with the script:
await page.goto(url, { waitUntil: 'networkidle0' })

Upvotes: 1

Related Questions