CODEWITHSUNDEEP
pythonnumpygoogle-app-engine
GRS
GRS

Reputation: 3084

Installing Numpy (v1.9.0), Pandas and SKlearn on GAE Standard Env (Python 27)

I've successfully ran my app using Webapp2 on local server. Now I wish to deploy this on Google App Engine. I'm currently debugging some errors since libraries are not compatible.

To configure the app, I followed the Google guide to add 3rd party plugins e.g. added lib folder and: 

# appengine_config.py
from google.appengine.ext import vendor

# Add any libraries install in the "lib" folder.
vendor.add('lib')

In the lib folder, I installed my requirements using pip install -t lib -r requirements.txt which is 

google-cloud-bigquery
oauth2client==4.1.2
google-api-python-client==1.6.4
pandas-gbq
pandas
scipy
scikit-learn==0.18.2
numpy==1.9.0

When I deploy my app, I see:

Traceback (most recent call last): File "/base/alloc/tmpfs/dynamic_runtimes/python27/54c5883f70296ec8_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle handler = _config_handle.add_wsgi_middleware(self._LoadHandler()) File "/base/alloc/tmpfs/dynamic_runtimes/python27/54c5883f70296ec8_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler handler, path, err = LoadObject(self._handler) File "/base/alloc/tmpfs/dynamic_runtimes/python27/54c5883f70296ec8_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject obj = import(path[0]) File "/base/data/home/apps/s~corded-epigram-579/gr:20171215t102242.406232341485344902/main.py", line 17, in import pandas as pd File "/base/data/home/apps/s~corded-epigram-579/gr:20171215t102242.406232341485344902/lib/pandas/init.py", line 19, in "Missing required dependencies {0}".format(missing_dependencies)) ImportError: Missing required dependencies ['numpy']

which basically tells me that it can't find numpy, but it is there in the lib folder.

I know that google supports numpy v1.6.1 which I can add using app.yaml file, but I need at least 1.9.0 for the pandas package.

Also added a screenshot of my main.py imports which I need to run the code.

enter image description here

Upvotes: 1

Views: 757

Answers (2)

Saim Abdullah
Saim Abdullah

Reputation: 304

add init.py file in your lib directory and then

appengine_config.py: 

from google.appengine.ext import vendor
import os
vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))#  vendor.add('lib')

then modify your imports to

from lib.something import something

Upvotes: -2

BrettJ
BrettJ

Reputation: 6841

App Engine standard environment only supports pure Python libraries, except for the built-in libraries provided by Google. Given that Numpy 1.6.1 doesn't work for you, you will need to consider using the flexible environment or anything platform like Compute Engine.

Depending on your application and its requirements, you might be able to architect your application into multiple App Engine "services". For example, say you needed to do some offline or asynchronous analysis using those libraries, you could run that code in a service that targets the flexible environment and have the rest of your application running in a service in the standard environment. The same project can split be split between environments or even languages by virtue of diving the application into services.

Upvotes: 3

Related Questions