CODEWITHSUNDEEP
pythongoogle-app-engine
tipsywacky
tipsywacky

Reputation: 3464

GAE python: What should I use for an operation that last for around 3 mins?

I need to get some data with app engine using urlfetch. The operation lasts for around 3 mins when doing it locally.

However, I've tried to use cron to do it but it returns errors with an F2 instance.

Traceback (most recent call last): File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 267

This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time.

So my question is what should I use to do this? Anyway I can fix it using cron or do I use module or taskqueue or background_thread to do it? Any advice will be appreciated.

Upvotes: 0

Views: 37

Answers (1)

Jaime Gomez
Jaime Gomez

Reputation: 7067

You can use the deferred library:

from google.appengine.ext import deferred

def get_data(a, b, c=None):
    logging.info("Doing something expensive!")
    # Do your work here

# Somewhere else
deferred.defer(do_something_expensive, "Hello, world!", 42, c=True)

By default you'll have 10 minutes to finish execution, but you could improve it if needed.

Also remember to add this to your app.yaml:

builtins:
- deferred: on

Upvotes: 1

Related Questions