CODEWITHSUNDEEP
pythonpandasgoogle-app-enginegoogle-cloud-platform
jfk83
jfk83

Reputation: 810

Can you install pandas in Google App Engine standard environment?

I am trying upload a project to Google App Engine Standard environment that uses pandas but I followed the instructions here:

https://cloud.google.com/appengine/docs/standard/python/tools/using-libraries-python-27

I am getting this error: ImportError: Missing required dependencies ['numpy']

I am not sure if I am on the right path.

Upvotes: 1

Views: 3799

Answers (4)

wescpy
wescpy

Reputation: 11167

(Nov 2021) Most of the answers here are outdated as this was "fixed" when the second-generation App Engine service launched in 2018. You can develop & deploy Python apps in a more idiomatic way on the platform now. You do have to use Python 3 however, as Python 2 is still on the legacy platform requiring pure Python code (except those w/built-in 3rd-party library support as mentioned elsewhere). Here is a simple Numpy and Pandas app I just deployed (3 files):

############
# app.yaml #
############

runtime: python39

###########
# main.py #
###########

from flask import Flask
import numpy as np
import pandas as pd

app = Flask(__name__)
@app.route('/')
def root():
    dates = pd.date_range("20211101", periods=7)
    df = pd.DataFrame(np.random.randn(7, 3),
            index=dates, columns=list("ABC"))
    return df.to_html()

####################
# requirements.txt #
####################

flask
pandas

Additional notes:

  1. You don't need to specify numpy in requirements.txt because it's a pandas dependency; having pandas is good enough.
  2. There are no more "blessed" 3rd-party libraries (w/or w/o C/C++ code) requiring that extra libraries section in app.yaml... just put what you want in requirements.txt.

Here is the output when hitting the App Engine page: enter image description here

Upvotes: 0

Marcin Trofimiuk
Marcin Trofimiuk

Reputation: 53

I confirm that it is possible to use numpy in appengine standard, but still pandas has quite a few references native libs.

So no pandas in appengine standard until it will be aded to list of native libs available through app.yaml import section.

Upvotes: 0

Thomas Korrison
Thomas Korrison

Reputation: 111

continuing on from Phillip Pearson, when developing locally you might run into "no module named _ctype" or "no module named _winreg" errors. You'll need to apply the work around below, assuming your using windows.

  • goto \google\appengine\tools\devappserver2\python\sandbox.py
  • find the definition of _WHITE_LIST_C_MODULES = [xxx] add following two lines to the list:
    '_winreg', '_ctypes',
  • try your app again.

Upvotes: 1

Phillip Pearson
Phillip Pearson

Reputation: 138

It looks like Pandas has a dependency on numpy, which includes some compiled C code and thus has to be loaded through App Engine's bundled third-party library system.

To fix this error, add numpy to your 'libraries' block in app.yaml:

libraries:
- name: numpy
  version: "1.6.1"

See Built-in Third-party Libraries for a full list of all the libraries you can include this way, and Using third-party libraries for more details on how the 'libraries' block works.

Note that if Pandas isn't itself pure Python, you still won't be able to install it on the Standard Environment. The above config will get numpy working for you though :)

Upvotes: 5

Related Questions