GAE import errors; sys.path shows wrong path for appengine libraries

Question

I'm developing a web app for Google App Engine in Python on Windows 10. Everything was working fine when my main.py was just serving templates.

import os
import urllib
from google.appengine.api import users
import jinja2
import webapp2

JINJA_ENVIRONMENT = jinja2.Environment(
    loader=jinja2.FileSystemLoader(os.path.dirname(__file__)),
    extensions=['jinja2.ext.autoescape'],
    autoescape=True)

But then I tried to add cloud storage, and got import errors no matter what library I tried. So I removed those references from main.py, and now I get an error with jinja2, which had been working fine!

ImportError: No module named jinja2

I don't remember everything I tried, but here's what I do know:

• jinja2 is installed; attempts to pip install/upgrade says it's already installed and up-to-date, and I see it in c:\python27\lib\site-packages.
• PYTHONPATH=C:\python27;c:\python27\lib;C:\Python27\DLLS for system and user.
• At one point I had installed GoogleAppEngineCloudStorageClient with PIP into my app's lib directory per this. It didn't work (the module failed to import), so I also tried adding sys.path.append(os.path.join(os.path.dirname(__file__), "lib")) but it didn't help. I think this is when I started getting the jinja2 import error. So I removed the statement from main.py. Still getting the jinja2 import error. I tried pip uninstall GoogleAppEngineCloudStorageClient but it said it wasn't installed, so I tried just deleting the lib directory. Still getting the jinja2 import error.
• Have tried restarting the service and rebooting my machine.

EDIT:

I stripped main.py all the way to the new project template,

import webapp2

class MainHandler(webapp2.RequestHandler):
    def get(self):
        self.response.write('Hello world!')

app = webapp2.WSGIApplication([
    ('/', MainHandler)
], debug=True)

and now I get an import error for webapp2:

ImportError: No module named webapp2

EDIT 2:

By inserting this at the top of my main.py,

import sys
print sys.path

I can see the wrong path for all the google packages:

'C:\\Program Files (x86)\\Google\\lib\\webapp2-2.5.2', 
'C:\\Program Files (x86)\\Google\\lib\\pycrypto-2.6', 
'C:\\Program Files (x86)\\Google\\lib\\jinja2-2.6', 
'C:\\Program Files (x86)\\Google\\lib\\markupsafe-0.15', 
'C:\\Program Files (x86)\\Google\\lib\\setuptools-0.6c11', 
'C:\\Program Files (x86)\\Google\\lib\\protorpc-1.0', 
'C:\\Program Files (x86)\\Google\\lib\\webob-1.1.1', 
'C:\\Program Files (x86)\\Google\\lib\\yaml-3.10'

They are actually in C:\Program Files (x86)\Google\google_appengine\lib

I don't know why I didn't have this problem before I tried to install that one package, but this may be related to a reported google issue.

Answer 1

Update: the issue was fixed in SDK version 1.9.40.

There is a GAE issue causing exactly this behaviour introduced in SDK version 1.9.37, see "ImportError: No module named webapp2" after Linux SDK upgrade (1.9.35 -> 1.9.38).

If your SDK version is 1.9.37 or 1.9.38 downgrade to 1.9.36, which you can find here. At least until the fix gets released.

Answer 2

Summary:

The webapp2 and jinja2 import errors are caused by sys.path corruption, a result of a GAE defect present in versions 1.9.37 or 1.9.38. It only impacts development; deployed versions should work. It can occur immediately upon upgrading or after attempting to install other items.

Solutions:

• Downgrading the AppEngineSDK to 1.9.36 worked for me.
• The GAE defect will be fixed at some point after 1.9.38. Check for Issue 12963 for status.
• Per Google issue 13084, another workaround is to manually patch sys.path in appengine_config.py. Docs here.