CODEWITHSUNDEEP
google-cloud-tasks
javi
javi

Reputation: 21

Google Cloud Tasks: some tasks remain in queue instead of being processed

I have a Google Cloud Task queue (rate: 10/s, bucket: 200, concurrent: 1) that dispatches tasks to a worker in a App Engine service (python 2.7 runtime) Tasks are normally added to the queue about 3-4/s. Each task is processed one at a time (no concurrency)

In general, each task is processed very fast(less than 1sg). Surprisingly, the queue sometimes randomly "pauses" a small subset of 5-20 tasks. New incoming tasks are processed as usual but those ones are blocked and stay on the queue for some minutes, even when worker is idle and might process them. After 7-9 minutes, they are processed automatically without any other interaction. Issue is this delay is too much and not acceptable :(

While "paused", I can manually execute those tasks by clicking on the "Run" button and they are immediately processed. So I'd discard some kind of limitation on the worker side.

I tried redeploying the queue.yaml. I also tried pausing and resuming the queue. Both with no effect.

No errors are notified. Tasks are not retried, just ignored for some minutes.

Has anybody experienced this behavior? Any help will be appreciated. Thanks.

Upvotes: 0

Views: 941

Answers (1)

Averi Kitsch
Averi Kitsch

Reputation: 921

Cloud Tasks now uses gcloud (Cloud SDK) to manage the queue configuration. queue.yaml is a part of the legacy App Engine SDK for App Engine Task Queues. Uploading a queue.yaml when using Cloud Tasks may cause your queue to be disabled or paused.

To learn more about queue management see, Using Queue Management versus queue.yaml.
To learn more about migrating from Task Queues to Cloud Tasks, see Migrating from Task Queues to Cloud Tasks

Upvotes: 0

Related Questions