CODEWITHSUNDEEP
google-app-enginetask-queue
John Wheeler
John Wheeler

Reputation: 797

How can I tell if a set of app engine tasks have all completed?

If I have a loop that enqueues say 100 tasks and each one of those tasks potentially enqueues a task, how can I tell if all tasks have completed?

I've thought about this problem using ShardedCounters. Once each task is completed, I could increment a counter then check to see if count == 100. Of course that falls apart with tasks spawning their own tasks unless I get into this recursive counting scenario. I'm not sure it's a good idea to go down that rabbit hole because it appears the sharded counters are not atomic.

Upvotes: 1

Views: 1264

Answers (2)

Daniel Roseman
Daniel Roseman

Reputation: 599698

You should take a look at the new AppEngine pipeline API. It's a way of chaining together expensive tasks, and notifying when they're complete. There's a also great video from the Google IO conference which demonstrates it.

Upvotes: 1

Cory Dolphin
Cory Dolphin

Reputation: 2670

Depending upon the scenario, you can test to see if any tasks (matching some pattern or having a certain ID/etc) are currently en-queued using the REST API:

http://code.google.com/appengine/docs/python/taskqueue/rest.html#method_taskqueue_tasks_list

Upvotes: 0

Related Questions