CODEWITHSUNDEEP
pythongoogle-app-engineurlfetchdeadlines
Yatharth Agarwal
Yatharth Agarwal

Reputation: 4564

GAE app raising DeadlineExceededError locally and DownloadError deployed

This is very weird. The title says most of it, my code should say the rest. Here's my main.py file:

from google.appengine.api import urlfetch
import webapp2
import jinja2

import json
import os

jinja_environment = jinja2.Environment(
    loader=jinja2.FileSystemLoader(os.path.dirname(__file__)))


class MainPage(webapp2.RequestHandler):
    def get(self):
        response = urlfetch.fetch("http://localhost:8080/api/helloworld?name=totty", method=urlfetch.GET)
        if response.status_code == 200:
            result = json.loads(response.content)

        template_values = {'response': result['msg']}
        template = jinja_environment.get_template('index.html')
        self.response.out.write(template.render(template_values))


app = webapp2.WSGIApplication(
    [('/', MainPage)],
    debug=True)

Here's my api.py file:

import webapp2
import json

class HelloWorld(webapp2.RequestHandler):
    def get(self):
        name = self.request.get('name') or 'world'
        msg = "Hello {}!".format(name)
        payload = json.dumps({'msg': msg})
        # payload = json.dumps({'dir': str(dir(self.request)), 'body': str(self.request.body), 'name': str(self.request.arguments())})

        self.response.headers['Content-Type'] = 'application/json'
        self.response.write(payload)

app = webapp2.WSGIApplication(
    [('/api/helloworld', HelloWorld)],
    debug=True)

And in case my app.yaml file would help:

application: stacksort
version: 1
runtime: python27
api_version: 1
threadsafe: true

handlers:
- url: /api/.*
  script: api.app
- url: /.*
  script: main.app

libraries:
- name: webapp2
  version: latest
- name: jinja2
  version: latest

Nothing changes even if I add deadline=30 to my urlfetch call. I tested the API using httpie and JQuery and it works perfectly fine and returns in under 5 seconds.

I looked at the other questions, but I'm still stumbling into the dark. Help, tips or refactoring would be appreciated.

I plan to add calls to the StackEchange Search API, so I suspect the problem might come in that time also. If there's a better way to do this, please tell. Thanks.

Upvotes: 0

Views: 240

Answers (2)

Tony Baguette
Tony Baguette

Reputation: 617

You are trying to fetch a URL to and from your application which is strongly discouraged on Google App Engine.

Locally you won't be able to call the development server because it serves only one request at a time. Multi-threading is not observed.

Note: the new experimental development server is now able to handle multiple requests at a time.

Multithreaded serving for better performance for complex applications and more correct semantics e.g. accessing your own application through urlfetch no longer deadlocks.

On production environment, GAE prevents the fetch service to call the same application.

To prevent an app from causing an endless recursion of requests, a request handler is not allowed to fetch its own URL. It is still possible to cause an endless recursion with other means, so exercise caution if your app can be made to fetch requests for URLs supplied by the user.

Upvotes: 1

Yatharth Agarwal
Yatharth Agarwal

Reputation: 4564

While upgrading the SDK, I noticed this addition to the DevServer page:

Note: dev_appserver.py can only serve one request at a time. If your application makes URL fetch requests to itself while processing a request, these requests will fail when using the development web server. (They will not fail when running on App Engine.) To test such requests, you can run a second instance of dev_appserver.py on a different port, then code your application to use the other server when making requests to itself.

So I guess that solves my problem (or at least gives a satisfactory explanation for it).

Upvotes: 0

Related Questions