CODEWITHSUNDEEP
pythonpython-3.xpython-requestspython-requests-html
tgraf2
tgraf2

Reputation: 167

Requests-html results in OSError: [Errno 8] Exec format error when calling html.render()

I am using requests-html and trying the render function, with little success. When I run this script using python3.8

#!/usr/bin/python3
from requests_html import HTML
file = "scrape/temp_file2.html"
with open(file) as html_file:
   source = html_file.read()
   html = HTML(html=source)
   html.render()
   match = html.find('#footer', first=True)
   try:
      print(match.hmtl)
   except:
      print('not found')

it results in a traceback:

python3 scrape/test1.py
Traceback (most recent call last):
  File "scrape/test1.py", line 10, in <module>
    html.render()
  File "/home/pi/.local/lib/python3.8/site-packages/requests_html.py", line 586, in render
    self.browser = self.session.browser  # Automatically create a event loop and browser
  File "/home/pi/.local/lib/python3.8/site-packages/requests_html.py", line 730, in browser
    self._browser = self.loop.run_until_complete(super().browser)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/pi/.local/lib/python3.8/site-packages/requests_html.py", line 714, in browser
    self._browser = await pyppeteer.launch(ignoreHTTPSErrors=not(self.verify), headless=True, args=self.__browser_args)
  File "/home/pi/.local/lib/python3.8/site-packages/pyppeteer/launcher.py", line 306, in launch
    return await Launcher(options, **kwargs).launch()
  File "/home/pi/.local/lib/python3.8/site-packages/pyppeteer/launcher.py", line 147, in launch
    self.proc = subprocess.Popen(  # type: ignore
  File "/usr/local/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/home/pi/.local/share/pyppeteer/local-chromium/588429/chrome-linux/chrome'

The chromium file is an executable:

ls -l /home/pi/.local/share/pyppeteer/local-chromium/588429/chrome-linux/chrome
-rwxr-xr-x 1 pi pi 214121928 Mar  9 17:21 /home/pi/.local/share/pyppeteer/local-chromium/588429/chrome-linux/chrome

If I remove the html.render() line it works fine (but won't render the javascript). Any ideas?

Upvotes: 6

Views: 2867

Answers (1)

tgraf2
tgraf2

Reputation: 167

After much searching I found a solution. The problem is that pyppeteer installs the x86-64 version of chromium on a 32-bit ARM processor. The solution below is to install the chromium files from Ubuntu and link to those files from the pyppeteer directory. From https://github.com/miyakogi/pyppeteer/issues/250:

install chromium-codecs-ffmpeg:
   https://launchpad.net/ubuntu/trusty/armhf/chromium-codecs-ffmpeg
   ➜ chromium-codecs-ffmpeg_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb
   sudo dpkg --force-all -i chromium-codecs-ffmpeg_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb

install chromium-browser:
   https://launchpad.net/ubuntu/trusty/armhf/chromium-browser
   ➜ chromium-browser_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb
   sudo dpkg --force-all -i chromium-browser_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb

Other notes:

  • make sure that you download a version >= 62. I've used version 65 and it works fine.
  • install chromium-codecs-ffmpeg first, then chromium-browser by using the command "dpkg --force-all -i".
  • after the installation of Chromium from Ubuntu packages, grab its path by running "whereis chromium-browser"
  • most probably, it's going to be on "/usr/bin/chromium-browser".
  • go to the directory where pyppeteer downloaded x86 Chromium on it.
  • in your case it is "/home/pi/.local/share/pyppeteer/local-chromium/588429/chrome-linux/".
  • delete all its files.
  • in the same directory. Create a symbol link which has the name "chrome" and points to Ubuntu's Chromium (you have got its path in step 3).
  • for example: ln -s /usr/bin/chromium-browser chrome
  • after performing those steps, everything should work fine.

Upvotes: 8

Related Questions