CODEWITHSUNDEEP
google-app-enginegoogle-cloud-datastoretask-queue
Lull
Lull

Reputation: 395

Why is GAE task on queue being run more often than specified?

I have a queue defined as follows:

....
<queue>
    <name>sendMailsBatch</name>
    <rate>1/m</rate>
    <max-concurrent-requests>1</max-concurrent-requests>
    <retry-parameters>
        <min-backoff-seconds>70</min-backoff-seconds>
        <max-doublings>1</max-doublings>
    </retry-parameters>
</queue>
....

I want there to be time gap of at least 60 seconds between each time a task runs. This must be the case no matter whether it is the same task being run because it fails, or whether it is different tasks being run.

The process starts by one task being put onto the queue, and this task will at the end - if all datastore operations are successfull - add another task to the queue (as it uses a cursor from the datastore operation executed by the task).

When I look at the log though, the tasks are executed too often: enter image description here

Why are the tasks executed this frequently, when I have configured that at most one task can run at a time, and at the most one task per minute, and if a task fails, there should be at least 70s between the runs?

Thanks, -Louise

Upvotes: 0

Views: 99

Answers (1)

dwelling
dwelling

Reputation: 491

When processing the queue, the app engine uses all of the concurrent requests specified to process what is already in its bucket. Once it finishes those tasks, it won't perform any additional work until a new task appears on the bucket. The rate at which these tasks are added to the bucket is defined by <rate>.

In your case, you set the <rate> correctly but since you didn't explicitly set the <bucket-size> parameter, it defaulted to 5 as mentioned here: https://cloud.google.com/appengine/docs/standard/java/config/queueref. Once you explicitly set the <bucket-size> to 1, you should no longer run into this issue.

Upvotes: 2

Related Questions