Error when Installing scrapy from zsh

Question

I had Python Scrapy on my Mac and everything was Ok til I update Bash to Zsh, Now I tried to install it with pip install scrapy but faced with

pip install Scrapy
Collecting Scrapy
  Downloading Scrapy-1.3.2-py2.py3-none-any.whl (239kB)
    100% |████████████████████████████████| 245kB 280kB/s
Requirement already satisfied: service-identity in /Library/Python/2.7/site-packages (from Scrapy)
Collecting parsel>=1.1 (from Scrapy)
  Downloading parsel-1.1.0-py2.py3-none-any.whl
Collecting six>=1.5.2 (from Scrapy)
  Downloading six-1.10.0-py2.py3-none-any.whl
Collecting lxml (from Scrapy)
  Downloading lxml-3.7.3.tar.gz (3.8MB)
    100% |████████████████████████████████| 3.8MB 41kB/s
Collecting Twisted>=13.1.0 (from Scrapy)
  Downloading Twisted-17.1.0.tar.bz2 (3.0MB)
    100% |████████████████████████████████| 3.0MB 134kB/s
Collecting cssselect>=0.9 (from Scrapy)
  Downloading cssselect-1.0.1-py2.py3-none-any.whl
Collecting PyDispatcher>=2.0.5 (from Scrapy)
  Downloading PyDispatcher-2.0.5.tar.gz
Collecting queuelib (from Scrapy)
  Downloading queuelib-1.4.2-py2.py3-none-any.whl
Collecting w3lib>=1.15.0 (from Scrapy)
  Downloading w3lib-1.17.0-py2.py3-none-any.whl
Requirement already satisfied: pyOpenSSL in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from Scrapy)
Requirement already satisfied: pyasn1 in /Library/Python/2.7/site-packages (from service-identity->Scrapy)
Requirement already satisfied: pyasn1-modules in /Library/Python/2.7/site-packages (from service-identity->Scrapy)
Requirement already satisfied: attrs in /Library/Python/2.7/site-packages (from service-identity->Scrapy)
Requirement already satisfied: zope.interface>=3.6.0 in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from Twisted>=13.1.0->Scrapy)
Collecting constantly>=15.1 (from Twisted>=13.1.0->Scrapy)
  Downloading constantly-15.1.0-py2.py3-none-any.whl
Collecting incremental>=16.10.1 (from Twisted>=13.1.0->Scrapy)
  Downloading incremental-16.10.1-py2.py3-none-any.whl
Collecting Automat>=0.3.0 (from Twisted>=13.1.0->Scrapy)
  Downloading Automat-0.5.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from zope.interface>=3.6.0->Twisted>=13.1.0->Scrapy)
Installing collected packages: six, w3lib, cssselect, lxml, parsel, constantly, incremental, Automat, Twisted, PyDispatcher, queuelib, Scrapy
  Found existing installation: six 1.4.1
    DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 778, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 754, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_uninstall.py", line 115, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 267, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/var/folders/rd/72lp2tzs02l049d0d33bzgxw0000gn/T/pip-DS44dD-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'

according to log I thought it because of six module try to install with

sudo pip install haxor-news --upgrade --ignore-installed six

it installed but when try to test it installed correctly, I typed scrapy and faced with

Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 7, in <module>
    from scrapy.cmdline import execute
  File "/Library/Python/2.7/site-packages/scrapy/__init__.py", line 34, in <module>
    from scrapy.spiders import Spider
  File "/Library/Python/2.7/site-packages/scrapy/spiders/__init__.py", line 10, in <module>
    from scrapy.http import Request
  File "/Library/Python/2.7/site-packages/scrapy/http/__init__.py", line 12, in <module>
    from scrapy.http.request.rpc import XmlRpcRequest
  File "/Library/Python/2.7/site-packages/scrapy/http/request/rpc.py", line 7, in <module>
    from six.moves import xmlrpc_client as xmlrpclib
ImportError: cannot import name xmlrpc_client

I find a question about this issue and it resolved with

sudo rm -rf /Library/Python/2.7/site-packages/six*
sudo rm -rf /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six*
sudo pip install six

but for sudo rm -rf /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six faced with

sudo rm -rf /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six*
rm: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info: Operation not permitted
rm: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six.py: Operation not permitted
rm: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six.pyc: Operation not permitted

also tried to update six with pip install --upgrade six but again error

pip install --upgrade six
Collecting six
  Using cached six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
  Found existing installation: six 1.4.1
    DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 778, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 754, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_uninstall.py", line 115, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 267, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/var/folders/rd/72lp2tzs02l049d0d33bzgxw0000gn/T/pip-_MQPsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'

where is the problem and how can I fix this issue?

Answer 1

you can also use this:

sudo -H pip install scrapy --ignore-installed six

This solve my issue. Hopefully it will help

Answer 2

I resolved problem with upgrading six module ,

sudo easy_install --upgrade six
export PYTHONPATH=/Library/Python/2.7/site-packages:$PYTHONPATH
source ~/.zshrc

resolved problem