Permission error with pip Python 3

Question

I am using Python 3 on Mac OS X. Everything was perfect, I used to install packages like this:

pip install somePackage

But IDLE stopped working so I had to re-install it. After that, when I ran pip, I got pip is not defined so I had to install pip. Now, every time I try to install a package using pip, I get this error:

Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 377, in move_wheel_files
    clobber(source, dest, False, fixer=fixer, filter=filter)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 287, in clobber
    ensure_dir(dest)  # common for the 'include' path
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 83, in ensure_dir
    os.makedirs(path)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 1] Operation not permitted: '/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/pygame'

I thought it may be from the version so I updated it. Now, whenever I run pip install something I get this error:

Command "/usr/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/bc/hwpn9sqn5352xcbst0p89snm0000gn/T/pip-build-hSNyxK/termcolor/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /var/folders/bc/hwpn9sqn5352xcbst0p89snm0000gn/T/pip-U3ZtNq-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/bc/hwpn9sqn5352xcbst0p89snm0000gn/T/pip-build-hSNyxK/termcolor/

I can't even delete a package - when I run pip uninstall something I get this error:

Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/uninstall.py", line 76, in run
    requirement_set.uninstall(auto_confirm=options.yes)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 346, in uninstall
    req.uninstall(auto_confirm=auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 754, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_uninstall.py", line 115, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 267, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/var/folders/bc/hwpn9sqn5352xcbst0p89snm0000gn/T/pip-OQKY_a-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy-1.8.0rc1-py2.7.egg-info'

My pip version is :

pip 9.0.1 from /Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg (python 2.7)

I am using Mac 10.12.6

How can I fix this?

Answer 1

The permission error you're seeing is due to the fact that your site-packages directory (where pip is trying to install to) is protected and owned by an administrator.

Where is the site packages directory

It depends on your Python distribution. That's why I asked for you to include the output of the $ which python command (from the command line), and the >>> import site; site.getsitepackages() command (from Python). These will help you figure out where your site packages directory is. Also, running pip -v will print the directory to which pip is trying to install.

It is important to note that Macs come with a system version of Python. You should not install any packages accessible to this system version of Python, because it is used by the system and you can potentially interfere with its normal workings and/or install malicious versions of packages that will do bad things when the system uses that Python (e.g., if someone made a nasty egg and called it "os" and put it in the site-packages directory for your system Python, then the next time the system python version ran "import os" it would import the nasty egg.)

Why overriding with sudo pip install is not a good solution

While running your pip install command as sudo would probably solve the problem, this is a short-term fix that is very bad for the reasons mentioned in my comment in @Nenand's answer - running pip install X downloads the egg for X from the internet, and runs python setup.py build && python setup.py install on whatever happens to be in the setup.py. While it's easy to be lulled into complacency because, hey, it works!, it's also naive to trust strangers on the internet (and let's face it, the people who make pip packages are strangers to you).

EDIT: I didn't even think of this, but there is also the potential that you mis-type a library, or guess the wrong name, and end up with one you REALLY didn't want (typosquatting). See this BleepingComputer article on typosquatting pip packages.

The proper solution

The proper solution is to figure out why your site-packages directory is protected, and either fix its permissions, or change the site-packages directory pip is using.

I suspect the reason you're seeing this permissions error is that pip is trying to install to the system Python's site-packages directory, which is owned by root for a good reason. The system Python is located at /usr/bin/ and its site-packages directory is located at the following directories:

'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages'
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/site-python'
'/Library/Python/2.7/site-packages'

My advice, in this case, is to install Python using Homebrew, Python.org, or Canopy, which will keep your snakes in separate cages and will leave the system Python alone. On my system, python is the system python, python2 is Python version 2, and python3 is Python version 3. This also makes it more clear which version I'm using. I also do not have a pip command - not on my path - so I can't accidentally install something to my system Python. I have to explicitly call pip2 or pip3. Using Homebrew Python will do all of this for you, so that's my most-favored recommendation of the many Python distributions available.