AWS Elastic Beanstalk failed to install Python package using requirements.txt Git Pip

Question

I tried to deploy a Flask app on AWS Elastic Beanstalk using eb deploy but failed.

I have the requirements.txt under the app directory:

Flask==0.12.2
numpy==1.13.3
pandas==0.21.1
requests==2.18.4
scipy==1.0.0
Werkzeug==0.12.2
-e git+http://github.com/hensing/PyDDE#egg=PyDDE

And python.config file under .ebextensions directory:

packages:
  yum:
    git: []
    gcc-c++: []
    make: []

The error message is:

INFO: Environment update is starting.

INFO: Deploying new version to instance(s).

ERROR: Your requirements.txt is invalid. Snapshot your logs for details.

ERROR: [Instance: i-03e92fa3c58b6e010] Command failed on instance. Return code: 1 Output: (TRUNCATED)... )

File "/usr/lib64/python2.7/subprocess.py", line 541, in check_call

raise CalledProcessError(retcode, cmd)

CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements .txt' returned non-zero exit status 2.

Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py failed. For more detail, check /var/log/eb-ac tivity.log using console or EB CLI.

INFO: Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].

ERROR: Unsuccessful command execution on instance id(s) 'i-03e92fa3c58b6e010'. Aborting the operation.

ERROR: Failed to deploy application.

And /var/log/eb-activity.log shows:

2018-01-19 04:26:53,878 ERROR Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 2

Traceback (most recent call last):

File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main

install_dependencies()

File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies

check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)

File "/usr/lib64/python2.7/subprocess.py", line 541, in check_call

raise CalledProcessError(retcode, cmd)

CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requiremen ts.txt' returned non-zero exit status 2 (Executor::NonZeroExitStatus)

It seems like this issue is because of the -e git+ installation is not supported by AWS Elastic Beanstalk?

Answer 1

This helped me:

I was able to fix this by adding enum34 = "==1.1.8" to pyproject.toml. Apparently enum34 had a feature in v1.1.8 that avoided this error, but this regressed in v1.1.9+. This is just a workaround though. The better solution would be for packages to use environment markers so you don't have to install enum34 at all unless needed.

Source: https://github.com/python-poetry/poetry/issues/1122

I created a file called 02_upgrade_pip.config in the .ebextensions folder to upgrade pip and install that specific version of enum34 every time a new instance is launched.

02_upgrade_pip.config

commands:
  01_remove_enum_34:
    command: pip uninstall -y enum34
    ignoreErrors: true
  02_install_enum_34:
    command: pip install enum34==1.1.8
    ignoreErrors: false
  03_pip_upgrade:
    command: /opt/python/run/venv/bin/pip install --upgrade pip
    ignoreErrors: false

Answer 2

The problem has been resolved.

It was NOT because of installation of PyDDE.

The actual reason was that installation of Scipy requires > 40MB memory and the default EC2 instance, t1.micro, doesn't have enough memory to install it. It can be resolved by using a larger EC2 instance. I eventually go with t2.medium.

Also, to install Pandas, it requires gcc. I modified .ebextensions\[env_name].config file with this: (I'm using python 2.7, from: elasticbeanstalk gcc and python-devel installation)

packages:
  yum:
    git: []
    gcc-c++: []
    python27-devel: []

Answer 3

You can try an alternate way of adding PyDDE to requirements

Flask==0.12.2
git+https://github.com/hensing/PyDDE.git