CODEWITHSUNDEEP
pythonseleniumselenium-webdriverpython-requestspython-unicode
Coolio2654
Coolio2654

Reputation: 1739

LookupError: unknown encoding: idna error with Selenium and Python

I was trying to get into introductory web scraping with selenium in Python, but I keep getting this mysterious error when I start up a basic Chrome instance:

LookupError: unknown encoding: idna

when using the following code:

from selenium import webdriver

url = 'http://www.webscrapingfordatascience.com/complexjavascript/'

driver = webdriver.Chrome()
driver.get(url)

I installed Chrome's necessary webdriver with brew cask install chromedriver.

I tried searching around the web for potential solutions, but it does not seem anyone else has asked this in relation to running selenium with Python, and this bug is also rather vague.

Edit

The question here's answer of doing an import encodings.idna gives me the new error of Module not found in Python.

In addition, my system is a mbp with Mac OS 10.11, Python is 3.7.2 (Clang 8.0.0), and pip is =19.0.3.

echo $PATH gives me the following output:

/Users/Michael/miniconda3/bin:/Users/Michael/intelpython3/bin:/Users/Michael/miniconda3/bin:/Users/Michael/miniconda3/bin:/opt/local/bin:/opt/local/sbin:/usr/local/sbin:/Users/Michael/anaconda3/lib/python3.6/site-packages:/Library/Frameworks/Python.framework/Versions/3.7/bin:/Users/Michael/anaconda3/bin:/Users/Michael/anaconda3/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin:/Library/TeX/texbin’:/Users/Michael/intelpython3/bin

Upvotes: 2

Views: 2680

Answers (1)

undetected Selenium
undetected Selenium

Reputation: 193108

This error message...

LookupError: unknown encoding: idna

...implies that there was an encoding / decoding error between idna and utf-8.

This error comes from _get_idna_encoded_host(host) method of models.py which is defined as follows:

@staticmethod
def _get_idna_encoded_host(host):
    import idna

    try:
        host = idna.encode(host, uts46=True).decode('utf-8')
    except idna.IDNAError:
        raise UnicodeError
    return host

A bit of your system details in terms of architecture and os would have helped us to debug your issue in a better way. However:

Solution

The solution is to add the following import:

import encodings.idna

Note: Ensure that pip is on the PATH and is 9.0.1 or better.

Upvotes: 1

Related Questions