CODEWITHSUNDEEP
pythongoogle-app-engine
user714852
user714852

Reputation: 2154

Appengine task runs for 5 seconds before throwing DeadlineExceededError

I have recently converted an app from using backends to modules. I was using backends for long running requests, primarily get requests to external url's (using the requests library). Since changing my app such that everything runs on the single default module I am getting DeadlineExceededError's. i.e.

DeadlineExceededError('The API call urlfetch.Fetch() took too long to respond and was cancelled.

I have read in the appengine documentation that there is a:

60-second deadline for HTTP requests, 10-minute deadline for tasks

The requests I am running are run as a task (using deferred.defer(...) ). However the task starts at, for example, 15:51:54.489 and finishes at 15:51:59.600 - a grand total of 5 seconds before throwing the error.

In app.yml automatic scaling enabled as follows:

automatic_scaling:
  min_idle_instances: 1
  max_idle_instances: 1
  max_pending_latency: 1s

It is not clear to me why this deadline error is being thrown. Could you please let me know how I can avoid throwing this error?

Upvotes: 1

Views: 1184

Answers (1)

user479870
user479870

Reputation:

As the exception mentions, the error is from urlfetch.Fetch, which every outbound HTTP request is routed through on App Engine, even if you use urllib, requests, or other libraries. And the default deadline for urlfetch is 5 seconds. So the problem is that your HTTP requests to external URLs are timing out. It seems unrelated to your switch from backends to regular instances. Probably a coincidence.

This page has an overview of the various types of errors behind DeadlineExceededError.

Upvotes: 3

Related Questions