CODEWITHSUNDEEP
python-2.7scrapy
Carter
Carter

Reputation: 143

Cannot Set Up a Scrapy Project

I type this into the terminal:

$ scrapy startproject tutorial

I cannot get it to start a new scrapy project, and I keep installing all the things scrapy needs. I just can't get it to work. This is the error message it gives me:

File "/Users/carterdavis/anaconda/bin/scrapy", line 4, in <module>
execute()
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/cmdline.py", line 122, in execute
cmds = _get_commands_dict(settings, inproject)
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/cmdline.py", line 46, in _get_commands_dict
cmds = _get_commands_from_module('scrapy.commands', inproject)
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/cmdline.py", line 29, in _get_commands_from_module
for cmd in _iter_command_classes(module):
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/cmdline.py", line 20, in _iter_command_classes
for module in walk_modules(module_name):
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/utils/misc.py", line 68, in walk_modules
submod = import_module(fullpath)
File "/Users/carterdavis/anaconda/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/commands/bench.py", line 3, in <module>
from scrapy.tests.mockserver import MockServer
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/tests/mockserver.py", line 6, in <module>
from twisted.internet import reactor, defer, ssl
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/twisted/internet/ssl.py", line 25, in <module>
from OpenSSL import SSL
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/OpenSSL/__init__.py", line 36, in <module>
from OpenSSL import crypto
ImportError: dlopen(/Users/carterdavis/anaconda/lib/python2.7/site-packages/OpenSSL/crypto.so, 10): Library not loaded: libssl.1.0.0.dylib
Referenced from: /Users/carterdavis/anaconda/lib/python2.7/site-packages/OpenSSL/crypto.so
Reason: image not found

I have Python 2.7 and every package necessary to run scrapy.

Upvotes: 2

Views: 1588

Answers (3)

Ravin Gupta
Ravin Gupta

Reputation: 853

This is a problem of cryptography installation, try installing crytography again, which includes installation of OpenSSL also. And your project will starts working with this. Follow these steps.

For Debian and Ubuntu

$ sudo apt-get install build-essential libssl-dev libffi-dev python-dev

For Fedora and RHEL-derivatives

$ sudo yum install gcc libffi-devel python-devel openssl-devel

After installing this, you should now be able to build and install cryptography with the usual

$ pip install cryptography

After this, try creating your scrapy project.

Upvotes: 0

Joe Hooper
Joe Hooper

Reputation: 816

Looks like you are using Anaconda - I had the same problem and this fixed it (tested on OS X 10.9 and 10.10).

First pip uninstall scrapy if you have it installed.

Use conda to install cryptography:

conda install cryptography

Set the DYLD_LIBRARY_PATH environment variable:

export DYLD_LIBRARY_PATH=$HOME/anaconda/lib

Then install scrapy again

pip install scrapy

Upvotes: 4

Wayne
Wayne

Reputation: 26

ImportError: dlopen(/Users/carterdavis/anaconda/lib/python2.7/site-packages/OpenSSL/crypto.so, 10):Library not loaded: libssl.1.0.0.dylib

According to the error message above, is there libssl package installed in your OS? If you are using Ubuntu(or Debian), try to execute there command:

apt-get install libssl1.0.0

Upvotes: 0

Related Questions