Html5Lib Installed in Python 3.5 but not 2.7

Question

I have been getting this nasty error:

Traceback (most recent call last):
  File "scrapeRecipe.py", line 29, in <module>
    br.select_form(name="aspnetForm")
  File "build/bdist.macosx-10.11-intel/egg/mechanize/_mechanize.py", line 619, in select_form
  File "build/bdist.macosx-10.11-intel/egg/mechanize/_html.py", line 260, in global_form
  File "build/bdist.macosx-10.11-intel/egg/mechanize/_html.py", line 267, in forms
  File "build/bdist.macosx-10.11-intel/egg/mechanize/_html.py", line 282, in _get_forms
  File "build/bdist.macosx-10.11-intel/egg/mechanize/_html.py", line 247, in root
  File "build/bdist.macosx-10.11-intel/egg/mechanize/_html.py", line 145, in content_parser
ImportError: No module named html5lib

I'm trying to understand why the Traceback is looking at these files. Moreover, when I do locate build/bdist.macosx-10.11-intel/egg/mechanize/_mechanize.py, it tells me that it doesn't exist.

I'm assuming I'm getting this module not found error because the html5lib module is located in /Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/_vendor/html5lib.

Question: Can we control where the traceback looks? Moreover, my sys.path is currently

['/Users/madelinezechar/eatLowCarbon', '/Library/Python/2.7/site-
packages/BeautifulSoup-3.2.1-py2.7.egg', '/Library/Python/2.7/site-
packages/html2text-2016.9.19-py2.7.egg', '/Library/Python/2.7/site-
packages/mechanize-0.3.1-py2.7.egg', '/Library/Python/2.7/site-
packages/requests-2.13.0-py2.7.egg', '/Library/Python/2.7/site-
packages/pip-9.0.1-py2.7.egg', 
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',     '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',     '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python',     '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',   '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',     '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', 
'/Users/madelinezechar/Library/Python/2.7/lib/python/site-packages',     '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC', 
'/Library/Python/2.7/site-packages']

I installed html5lib with the command easy_install html5lib. When I ran it, I get this message:

Adding html5lib 0.999999999 to easy-install.pth file

Using /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages
Processing dependencies for html5lib
Finished processing dependencies for html5lib

I am concerned about it using python 3.5 when my sys.path is looking at 2.7 an dd my python code runs on 2.7. Also, my easy_install.pth doesn't appear to have html5lib, even after installation:

import sys; sys.__plen = len(sys.path)

 ./BeautifulSoup-3.2.1-py2.7.egg
   ./html2text-2016.9.19-py2.7.egg
   ./mechanize-0.3.1-py2.7.egg
   ./requests-2.13.0-py2.7.egg
   ./pip-9.0.1-py2.7.egg
   import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

When I run pip3 freeze, I see html5lib listed. When I run pip freeze, I do not. If I try sudo pip install html5lib, I get this ugly message:

What does this error message mean? How can pip freeze not return html5lib, when I know that html5lib.py is in Python/2.7/site-packages/bs4/builder ?

Answer 1

I'm getting a different error, but to install html5lib for python 2.7, this is my solution:

pip install --ignore-installed six --user
sudo -H pip install html5lib --ignore-installed

To learn more, this is an excellent thread: https://github.com/pypa/pip/issues/3165