CODEWITHSUNDEEP
pythongoogle-app-engine
Justin
Justin

Reputation: 4843

Task scheduling in AppEngine dev_appserver.py

I have a [python] AppEngine app which creates multiple tasks and adds them to a custom task queue. dev_appserver.py seems to ignore the rate/scheduling parameters I specify in queue.yaml and executes all the tasks immediately. This is a problem [as least for dev/testing purposes] as my tasks call a rate-throttled url; immediate execution of all tasks breaches the throttling limits and returns me a bunch of errors.

Does anyone know if task scheduling if dev_appserver.py is disabled ? I can't find anything that suggests this in the AppEngine docs. Can anyone suggest a workaround ?

Thank you.

Upvotes: 5

Views: 1057

Answers (3)

Tim Dierks
Tim Dierks

Reputation: 2251

The documentation lies: the development server doesn't appear to support rate limiting. (This is documented for the Java dev server, but not for Python). You can demonstrate this by pausing a queue by giving it a 0/s rate, but you'll find it executes tasks anyway. When such an app is uploaded to production, it behaves as expected.

I opened a defect.

Upvotes: 1

Eren Güven
Eren Güven

Reputation: 2374

Rate parameter is not used for setting absolute upper bounds of TaskQueue processing. In fact, if you use for example:

rate: 10/s
bucket_size: 20

the processing can burst up to 20/s. Something more useful would be:

max_concurrent_requests: 1

which sets the maximum number of execution to 1 at a time.

However, this will not stop tasks from executing. If you are adding multiple Tasks a time but know that they need to be executed at a later time, you should probably use countdown.

_countdown using deferred library
countdown using Task class

Upvotes: 0

systempuntoout
systempuntoout

Reputation: 74064

When your app is running in the development server, tasks are automatically executed at the appropriate time just as in production.
You can examine and manipulate tasks from the developer console: http://localhost:8080/_ah/admin/taskqueue

Documentation here

Upvotes: 4

Related Questions