CODEWITHSUNDEEP
pythongoogle-app-enginetask-queueurlfetch
bscott
bscott

Reputation: 532

App Engine Python UrlFetch.set_default_fetch_deadline

I have looked through the docs here: https://cloud.google.com/appengine/docs/python/urlfetch/ and here: https://cloud.google.com/appengine/articles/deadlineexceedederrors?hl=en

I also found this stack overflow question which relates to my question: How to set timeout for urlfetch in Google App Engine?

I am connecting from my app engine app to an outside web service, that I do not have control over. Sometimes the requests take longer than 60 seconds. I set my application up to use the deferred app engine task queue api.

I am so confused. In the docs I've read it seems as though urlfetch has a maximum deadline of 60 seconds. But if its running in a task_queue it's 10 minutes? I really just need someone to clarify this for me.

Does that mean the task has 10 minutes to complete, but the urlfetch inside the task is still limited to the 60 seconds?

Pseudocode:

myTask = newTask()
deffered.defer(myTask.long_process, _queue="myqueue")

class newTask:
    url = "https://example.com"
def long_process(self):
    #will setting the deadline to more than 60 seconds work or not? 
    urlfetch.set_default_fetch_deadline(120)
    data = {}
    resp = urlfetch.fetch(self.url, method="POST", payload=data)
    #do something with resp....

Upvotes: 2

Views: 2763

Answers (2)

user2091046
user2091046

Reputation: 625

You can control both the urlfetch and the deferred task deadline.

Both can run for up to 600s AFAIK.

The one thing you shoudn't do is setting the urfetch deadline to a higher value than the task ;)

Upvotes: 0

Dan Cornilescu
Dan Cornilescu

Reputation: 39824

You're on the right track. Tiny correction: there is no 60s max for urlfetch.set_default_fetch_deadline(), it you might have been misled by the context of the discussion.

You can bump the 120 value up to 600, see the OP's comment to the selected answer in this recent Q&A: Appengine task runs for 5 seconds before throwing DeadlineExceededError

Upvotes: 4

Related Questions