Can't install scrapy with Python 3?

Question

I am using Python 3.6.3 and Pip 9.0.1 but still can't install scrapy? I am doing this on windows. When executing the following command

pip3 install scrapy

I am greeted with this error first..

----------------------------------------
Failed building wheel for Twisted
Running setup.py clean for Twisted
Failed to build Twisted
Installing collected packages: Twisted, scrapy
Running setup.py install for Twisted ... error

Then it continues, the second error stops it completly and seems a lot more fatal...

Command "c:\users\admin\appdata\local\programs\python\python36-32\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\admin\\AppData\\Local\\Temp\\pip-build-zxkenzjd\\Twisted\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\admin\AppData\Local\Temp\pip-tr72roue-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\admin\AppData\Local\Temp\pip-build-zxkenzjd\Twisted\

I have tried executing the following commands as suggested on this answer:

pip install -U setuptools
pip install -U wheel

Answer 1

for my env: Win11 x64 + Python 3.11 x64

Solution:

• downloaded related whl
  • https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
    • Click: twisted_iocpsupport‑1.0.2‑cp311‑cp311‑win_amd64.whl
      • note
        • cp311 = CPython 3.11
        • amd64 = x64
      • download: https://download.lfd.uci.edu/pythonlibs/archived/twisted_iocpsupport-1.0.2-cp311-cp311-win_amd64.whl
• install whl
  • pip install twisted_iocpsupport-1.0.2-cp311-cp311-win_amd64.whl

Answer 2

I had the same problem too but I solved it as follow:

Open the Anaconda Prompt as administrator (For Windows10: open cortana/search Anaconda Prompt/choose Run as Administrator)

You should go to the path of Anaconda, for me was like:

C:\WINDOWS\system32>cd ..
C:\WINDOWS>cd..
C:\>cd  ProgramData
C:\ProgramData>cd Anaconda3
C:\ProgramData>Anaconda3>

Then you should run the following command

C:\ProgramData>Anaconda3>conda install -c anaconda twisted

At some point it asks

Proceed ([y]/n)?

type y. Now twisted is installed.

To install scrapy, you either install it in Anaconda Prompt (as administrator) by running the following command:

C:\ProgramData>Anaconda3>conda install -c conda-forge scrapy

(again y for Proceed ([y]/n)?)

or on jupyter notebook and run the command

!pip install scrapy