Jupyter cannot start, "ImportError: This package should not be accessible"

Question

I am unable to start a Jupyter notebook server on a linux machine. When I type jupyter notebook, I get the following errors:

ImportError: This package should not be accessible on Python 3. Either you are trying to run from the python-future src folder or your installation of python-future is corrupted.
During handling of the above exception, another exception occurred:
ModuleNotFoundError: No module named 'urlparse'

Here is some system info, let me know if anything else might be helpful:

$ jupyter --version
4.4.0

$ which jupyter
/usr/bin/jupyter

$ which python
/usr/bin/python

$ which python3
/usr/bin/python3

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"

$ cat /proc/version
Linux version 4.15.0-38-generic (buildd@lcy01-amd64-023) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #41-Ubuntu SMP Wed Oct 10 10:59:38 UTC 2018

$ pip3 --version
pip 9.0.1 from /usr/local/lib/python3.6/dist-packages (python 3.6)

When I try python2 -m pip install urlparse --user, I get an error: Collecting urlparse Could not find a version that satisfies the requirement

The reported error resembles the one shown here, but the stated solution of creating and using a python3 virtual environment doesn't seem to immediately fix the jupyter notebook command, although it does replace python and python3 executable with links to the virtual environment:

(env) $ which jupyter
/usr/bin/jupyter
(env) $ which python
/home/.../env/bin/python
(env) $ which python3
/home/.../env/bin/python3

In case there is some question about the python path, here is the path for python 2:

$ ipython
...
In [2]: sys.path
Out[2]: 
['/usr/lib/python2.7',
 '/usr/lib/python2.7/plat-x86_64-linux-gnu',
 '/usr/lib/python2.7/lib-tk',
 '/usr/lib/python2.7/lib-old',
 '/usr/lib/python2.7/lib-dynload',
 '/home/$USER/.local/lib/python2.7/site-packages',
 '/usr/local/lib/python2.7/dist-packages',
 '/usr/lib/python2.7/dist-packages',
 '/usr/lib/python2.7/dist-packages/gtk-2.0',
 '/home/$USER/.local/lib/python2.7/site-packages/IPython/extensions',
 '/home/$USER/.ipython']

And for python3

$ ipython3
...
In [2]: sys.path                                                                                                                                 
Out[2]: 
['/usr/lib/python36.zip',
 '/usr/lib/python3.6',
 '/usr/lib/python3.6/lib-dynload',
 '/home/$USER/.local/lib/python3.6/site-packages',
 '/usr/local/lib/python3.6/dist-packages',
 '/usr/lib/python3/dist-packages',
 '/home/$USER/.local/lib/python3.6/site-packages/IPython/extensions',
 '/home/$USER/.ipython']

The full stack trace of the first error when starting Jupyter is:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 18, in <module>
    from urllib.request import urlretrieve
  File "/usr/lib/python3.6/urllib/request.py", line 88, in <module>
    import http.client
  File "/home/$USER/.local/lib/python3.6/site-packages/http/__init__.py", line 7, in <module>
    raise ImportError('This package should not be accessible on Python 3. '
ImportError: This package should not be accessible on Python 3. Either you are trying to run from the python-future src folder or your installation of python-future is corrupted.

The contents of /usr/bin/jupyter are as follows:

$ cat `which jupyter`
#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'jupyter-core==4.4.0','console_scripts','jupyter'
__requires__ = 'jupyter-core==4.4.0'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('jupyter-core==4.4.0', 'console_scripts', 'jupyter')()
    )

One suggested answer is that there is a spurious http library, "it happens to be overridden by some 3rd-party module "http" at /home/$USER/.local/lib/python3.6/site-packages/http". When I remove this library manually, I now get an error importing the site module, as follows:

$ cd /home/$USER/.local/lib/python3.6/site-packages
$ mv ./http ~/Desktop/python_disabled/python3/
$ jupyter notebook
Error processing line 1 of /home/$USER/.local/lib/python3.6/site-packages/lazr.restfulclient-0.14.0-py3.6-nspkg.pth:

Failed to import the site module
Traceback (most recent call last):
  File "/usr/lib/python3.6/site.py", line 174, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.6/types.py", line 171, in <module>
    import functools as _functools
  File "/usr/lib/python3.6/functools.py", line 21, in <module>
    from collections import namedtuple
  File "/usr/lib/python3.6/collections/__init__.py", line 32, in <module>
    from reprlib import recursive_repr as _recursive_repr
  File "/home/$USER/.local/lib/python3.6/site-packages/reprlib/__init__.py", line 7, in <module>
    raise ImportError('This package should not be accessible on Python 3. '
ImportError: This package should not be accessible on Python 3. Either you are trying to run from the python-future src folder or your installation of python-future is corrupted.

Answer 1

This part is wrong:

  File "/usr/lib/python3.6/urllib/request.py", line 88, in <module>
    import http.client
  File "/home/$USER/.local/lib/python3.6/site-packages/http/__init__.py", line 7, in <module>
    raise ImportError('This package should not be accessible on Python 3. '

A standard module tries to import http.client, also a standard module. And in your installation, it happens to be overridden by some 3rd-party module "http" at /home/$USER/.local/lib/python3.6/site-packages/http.

Remove that 3rd-party module.

---

Judging by the same problem with reprlib package after uninstalling http in your updated question, you seem to have the micropython-lib set of packages installed in /home/$USER/.local/lib/python3.6. Since there are lots of them, it's easier to just delete the entire directory. I've no idea how they ended up there since they are not only incompatible with a regular CPython installation but with Python 3 in general.

---

Finally, note that depending on your needs, using virtualenv may be a more manageable solution than pip --user.