CODEWITHSUNDEEP
pythonpython-3.xpython-requestsmultiprocessingpython-requests-html
S. Allen
S. Allen

Reputation: 171

Python requests-html throwing multiprocessing error

I am attempting to move over my web scraping program from using the requests library to the requests-html library to allow me to render the javascript on webpages. On the import of the module this error is thrown:

Traceback (most recent call last):
  File "backend2.py", line 2, in <module>
    import requests_html
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests_html.py", line 9, in <module>
    import pyppeteer
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/__init__.py", line 30, in <module>
    from pyppeteer.launcher import connect, launch, executablePath  # noqa: E402
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/launcher.py", line 24, in <module>
    from pyppeteer.browser import Browser
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/browser.py", line 15, in <module>
    from pyppeteer.page import Page
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/page.py", line 20, in <module>
    from pyppeteer.coverage import Coverage
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/coverage.py", line 15, in <module>
    from pyppeteer.util import merge_dict
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/util.py", line 10, in <module>
    from pyppeteer.chromium_downloader import check_chromium, chromium_executable
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/chromium_downloader.py", line 15, in <module>
    from tqdm import tqdm
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tqdm/__init__.py", line 1, in <module>
    from ._tqdm import tqdm
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tqdm/_tqdm.py", line 75, in <module>
    mp_lock = mp.RLock()  # multiprocessing lock
AttributeError: module 'multiprocessing' has no attribute 'RLock'

Any help is appreciated!

Upvotes: 2

Views: 515

Answers (1)

Rob Bricheno
Rob Bricheno

Reputation: 4653

You are using Python 3.7, but the github of requests-html states that only Python 3.6 is supported (bottom of page). I get all kinds of horrible errors when I try to use Python 3.7 but 3.6 works fine. So, seems strange I know, but please try using 3.6

Even so, as @georgexsh states, there should still be a RLock available for import from multiprocessing in 3.7 so it looks like your error may actually be that one of your code files is called multiprocessing.py or you have your own package called multiprocessing (so you are importing that instead of the module you wanted).

Upvotes: 1

Related Questions