CODEWITHSUNDEEP
pythonamazon-web-servicesnumpyamazon-ec2amazon-elastic-beanstalk
sometimesiwritecode
sometimesiwritecode

Reputation: 3223

Scipy seems impossible to get working on Beanstalk

I have been trying to get scipy working on a Beanstalk application. Basically I have a python web app I am developing that uses Flask and a few other libraries including Scipy. In my project directory I have only 3 files. They are:

application.py
requirements.txt
.ebextensions/python.config

My application.py file is very simple and doesn't contain much of substance that could be causing my problems so will not paste the code in it here. My requirements.txt file contains the following:

Flask==0.10.1
itsdangerous==0.24
Jinja2==2.9.6
MarkupSafe==1.0
numpy==1.12.1
scipy==0.19.0
scikit-learn==0.18.1
sklearn==0.0
Werkzeug==0.12.2

and my .ebextensions/python.config file contains the following:

packages:  
  yum:
    make: []
    gcc-c++: []
    gcc-gfortran: []
    python-devel: []
    atlas-sse3-devel: []
    lapack-devel: []
    libpng-devel: []
    freetype-devel: []
    zlib-devel: []
container_commands:  
  AddGlobalWSGIGroupAccess: 
    command: "if ! grep -q 'WSGIApplicationGroup %{GLOBAL}' ../wsgi.conf ; then echo 'WSGIApplicationGroup %{GLOBAL}' >> ../wsgi.conf; fi;"

I basically just set up the flask app and get it working locally before using the Elastic Beanstalk cli on my mac to try and send my local app to Beanstalk and have it run as a website. So after i have done all the pip installs and an eb init I do the following in my terminal:

eb create flask-env

This runs for a bit but then fails giving the following error:

ERROR: Your requirements.txt is invalid. Snapshot your logs for details.
ERROR: [Instance: i-00b93584dae09f4d2] Command failed on instance. Return code: 1 Output: (TRUNCATED)...)
  File "/usr/lib64/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)

I then check my eb-activity.log file to try and get a better understanding of the problem. In the eb-activity.log is so much stuff that looks like it has to do with this error but some of the most helpful error messages in the log file are the following:

File "scipy/linalg/setup.py", line 20, in configuration
          raise NotFoundError('no lapack/blas resources found')
      numpy.distutils.system_info.NotFoundError: no lapack/blas resources found

 lapack_info:
        libraries lapack not found in ['/opt/python/run/venv/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib']
        NOT AVAILABLE

openblas_lapack_info:
        libraries openblas not found in ['/opt/python/run/venv/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib']
        NOT AVAILABLE

As you can tell these error messages in the logs are pretty verbose and hard to interpret. After googling around I thought my problem was stemming from scipy (and perhaps numpy) having a bunch of dependencies that are not Python related which pip does not handle.

I thought based on some other answers i found that my .ebextensions/python.config would fix this problem but it does not seem to - nor does it even appear in my directory as it starts with a '.'. Am I doing something small slightly wrong or what?

Update: I am on mac OS if that matters too. I found some stuff that sort of looked helpful at the link suggested in the comments but this seems out dated (as per the comments on the answer) and not specific to mac OS and thus seems kind of useless.

Upvotes: 1

Views: 579

Answers (1)

Aidan Hoolachan
Aidan Hoolachan

Reputation: 2379

You're just missing a few dependencies. Try using homebrew for development on your mac:

brew install numpy --with-openblas
brew install scipy --with-openblas

Theano has similar dependencies and provides a few methods for installation in various environments: http://deeplearning.net/software/theano/install_macos.html

Upvotes: 1

Related Questions