Problems with upgrading pip in Homebrew Python 2.7 installation

Question

I've got Python 2.7.9 installed using Homebrew on my Mac, and Homebrew also installs pip. There's also an older, unused version of Python that was installed by default on my Mac.

The problem is that when I try to upgrade pip (using pip install --upgrade pip), pip seems to want to upgrade the version of pip that comes with the older, default version of Python. Here's what happens (after doing a fresh install of Python 2.7.9, i.e., brew remove python followed by brew install python):

$ pip -V
pip 6.0.7 from /usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-6.0.7-py2.7.egg (python 2.7)

The above seems correct. However, the following happens:

$ pip install --upgrade pip
You are using pip version 6.0.7, however version 6.0.8 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-6.0.8-py2.py3-none-any.whl#md5=41e73fae2c86ba2270ff51c1d86f7e09
  Using cached pip-6.0.8-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 6.0.7
    Uninstalling pip-6.0.7:
      Successfully uninstalled pip-6.0.7

Successfully installed pip-1.4.1

Why was pip 1.4.1 installed? Now I get:

$ pip -V
pip 1.4.1 from /Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg (python 2.7)

That seems to correspond to the older default version of Python. When I try to upgrade again, it fails:

$ pip install --upgrade pip
Downloading/unpacking pip from https://pypi.python.org/packages/source/p/pip/pip-6.0.8.tar.gz#md5=2332e6f97e75ded3bddde0ced01dbda3
  Downloading pip-6.0.8.tar.gz (1.2MB): 1.2MB downloaded
  Running setup.py egg_info for package pip

    warning: no previously-included files found matching '.coveragerc'
    warning: no previously-included files found matching '.mailmap'
    warning: no previously-included files found matching '.travis.yml'
    warning: no previously-included files found matching 'pip/_vendor/Makefile'
    warning: no previously-included files found matching 'tox.ini'
    warning: no previously-included files found matching 'dev-requirements.txt'
    no previously-included directories found matching '.travis'
    no previously-included directories found matching 'docs/_build'
    no previously-included directories found matching 'contrib'
    no previously-included directories found matching 'tasks'
    no previously-included directories found matching 'tests'
Installing collected packages: pip
  Found existing installation: pip 1.4.1
    Uninstalling pip:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/commands/install.py", line 241, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1294, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 525, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1639, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/pip/util.py", line 294, in renames
    shutil.move(old, new)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 300, in move
    rmtree(src)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 247, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 252, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 250, in rmtree
    os.remove(fullname)
OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/pip-1.4.1-py2.7.egg/EGG-INFO/dependency_links.txt'

Storing complete log in /Users/[me]/.pip/pip.log

However, pip installs correctly to /usr/local/lib/python2.7/site-packages (the Homebrew-provided version), and I have no apparent issues with using pip to install modules and using the modules. Here's the output of which, after all of the above:

$ which pip
/usr/local/bin/pip
$ which python
/usr/local/bin/python

What's going on? Any help would be appreciated.

Answer 1

I had this problem too (Homebrew Python 2.7.9 on Yosemite). I ran it into trying to do pip install virtualenv:

> which pip2.7
/usr/local/bin/pip2.7
> which python
/usr/local/bin/python
> pip install virtualenv
Requirement already satisfied (use --upgrade to upgrade): virtualenv in /Library/Python/2.7/site-packages
> pip install --upgrade virtualenv
..
OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/virtualenv-1.11.5-py2.7.egg-info/dependency_links.txt'

I solved it by hiding the files in /Library/Python/2.7/site-packages temporarily. And moving them back when I was finished.

> pip install virtualenv
Collecting virtualenv
  Using cached virtualenv-12.0.7-py2.py3-none-any.whl
Installing collected packages: virtualenv

Successfully installed virtualenv-12.0.7
> which virtualenv
/usr/local/bin/virtualenv
>

It works properly too. I get 2.7.9 inside the virtualenv and can install numpy, matplotlib, scipy, and they work.

Answer 2

I think this is the same issue described here: https://github.com/pypa/pip/issues/2319

It's caused by pip misbehaving when there's another pip egg in sys.path.

The easiest fix is to keep running sudo python -m pip uninstall pip until that stops working, and then brew postinstall python (which reinstalls pip).