CODEWITHSUNDEEP
pythongoogle-app-enginetask-queue
Paolo177
Paolo177

Reputation: 366

Google App Engine - Task Queue - How to run a task if a condition is met

I have simple python code that creates and handles push queues in google app engine.

To create tasks I use:

 taskqueue.Queue(MY_QUEUE_NAME).add(taskqueue.Task(url=MY_URL,params=MY_PARAMS))

To handle task I use:

class QueueHandler(webapp2.RequestHandler):
    def post(self):
        params = self.request.get('params')
        MY_FUNCION(params)

This works great but I need to run my function only if a condition is met! I would like to do something like this:

class QueueHandler(webapp2.RequestHandler):
    def post(self):
        params = self.request.get('params')
        if IS_RAINING:
            MY_FUNCION(params)
        ELSE:
            RETRY_ANOTHER_TIME

Right now instead of RETRY_ANOTHER_TIME, I used this:

raise ValueError("not_raining")

but I know this is a very bad way to do that.

I also imagined that I could use instead of RETRY_ANOTHER_TIME:

taskqueue.Queue(MY_QUEUE_NAME).add(taskqueue.Task(url=MY_URL,params=MY_PARAMS))

creating another task that will try the same thing but this is slow and awful to me considering this way I could create many duplicates useless tasks.

Upvotes: 1

Views: 342

Answers (1)

Dan Cornilescu
Dan Cornilescu

Reputation: 39824

You don't really have other significantly different alternatives.

There is no way to execute the task conditionally, the execution is triggered only by the task's ETA. So your only option is to check the condition inside the task itself - just as you do now - and do the job or not. But by that time the task is already running.

Once a task is scheduled to run, it's either:

  • successful (if the handler returns a code between 200 and 299) and taken off the queue. The only way to keep the task on the queue to RETRY_ANOTHER_TIME is to enqueue another task (a clone of itself, if you want) - your second suggestion
  • failed, in which case it's kept on the queue and retried, but according to the task retry policy configuration, which by default is a gradual backoff scheme. This is effectively what you did when you raised the ValueError exception. Maybe a quieter/slightly more graceful solution would be to simply return a code outside of the 200-299 "OK" range and tune the retry policy according to your needs.

I favour the 1st approach - keep enqueuing and consuming tasks for as long as you need to "execute MY_FUNCION if IS_RAINING". They're pretty light.

If you need to "execute MY_FUNCION if IS_RAINING" (al)most all the time and the interval between the IS_RAINING check is longer than 1 minute and you can obtain the parameters that you're currently passing to the task in some other way it might be more convenient to use a cron job instead of the push queue. Lots of ifs.

Upvotes: 3

Related Questions