CODEWITHSUNDEEP
pythonpython-3.xweb-scrapingpython-requests-htmlpyppeteer
m_ocean
m_ocean

Reputation: 355

Why do I get pyppeteer.errors.PageError when using requests_html?

I'm scraping a list of similar webpages and sometimes get an error (see at the end).

The code I use:

from requests_html import HTMLSession    
import pyppdf.patch_pyppeteer

link = 'https://www.wildberries.ru/catalog/1588749/detail.aspx?targetUrl=BP'
# It's always a different link from the list, but here I simplified it.

session = HTMLSession()
resp = session.get(link)
resp.html.render()

Most pages don't cause the error, but some rare ones do. The error appears either on resp = session.get(link) or resp.html.render(). Here it is:

Traceback (most recent call last):
  File "/Users/max/Dropbox/WORK/projects/wildberries_parser/parsers/catalog_parser_3.py", line 133, in <module>
    row = parse_item_page(link)
  File "/Users/max/Dropbox/WORK/projects/wildberries_parser/parsers/catalog_parser_3.py", line 36, in parse_item_page
    resp.html.render()
  File "/Users/max/opt/anaconda3/envs/wildberries_parser/lib/python3.6/site-packages/requests_html.py", line 598, in render
    content, result, page = self.session.loop.run_until_complete(self._async_render(url=self.url, script=script, sleep=sleep, wait=wait, content=self.html, reload=reload, scrolldown=scrolldown, timeout=timeout, keep_page=keep_page))
  File "/Users/max/opt/anaconda3/envs/wildberries_parser/lib/python3.6/asyncio/base_events.py", line 488, in run_until_complete
    return future.result()
  File "/Users/max/opt/anaconda3/envs/wildberries_parser/lib/python3.6/site-packages/requests_html.py", line 512, in _async_render
    await page.goto(url, options={'timeout': int(timeout * 1000)})
  File "/Users/max/opt/anaconda3/envs/wildberries_parser/lib/python3.6/site-packages/pyppeteer/page.py", line 856, in goto
    raise PageError(result)
pyppeteer.errors.PageError: net::ERR_NAME_NOT_RESOLVED at https://www.wildberries.ru/catalog/1588749/detail.aspx?targetUrl=BP

I can't understand it and haven't figured this out on my own. Can you tell me, what's going on?

Upvotes: 0

Views: 2205

Answers (1)

Mattwmaster58
Mattwmaster58

Reputation: 2576

ERR_NAME_NOT_RESOLVED means there was a problem resolving the IP address from the name. This could be a problem with your computer, your router, or your DNS resolver.

You may want to try changing your DNS provider to Google's (8.8.8.8 & 8.8.4.4).

Upvotes: 1

Related Questions