CODEWITHSUNDEEP
google-app-enginegoogle-cloud-platformgoogle-cloud-tasks
smorhaim
smorhaim

Reputation: 798

Google Cloud Task Queue Consuming Too Slow

I have set up a Google Cloud Task queue and sent thousands of tasks. I have modified the settings but nothing makes it go any faster.

The task is a simple POST request to my app engine server which returns in 10 seconds.

I've tried adding more target servers in App Engine, but obviously that is not the problem since if I call my target server directly, it handles much more than 40-50 requests in under 10 seconds.... but the queue seems to only process 5-10 per minute.

Checking the app engine server is underutilized, 20% CPU, very low ram, very low traffic, so it is obviously not getting as many tasks as it should from the queue.

So how can I make the tasks be dispatched faster?

My queue yaml with the latest settings. I've tried many variations of it... 

- name: combinedexpress1
  rate: 120/m
  max_concurrent_requests: 70
  bucket_size: 150
  retry_parameters:
    task_retry_limit: 2
    task_age_limit: 1d

Upvotes: 0

Views: 518

Answers (1)

minou
minou

Reputation: 16563

In my experience, setting max_concurrent_requests seems to really slow down task processing though I don't have any idea why.

Try using default settings and see if that speeds up task processing. If it does, then you can iteratively add the settings that are important and make sure they don't slow down task processing.

Upvotes: 1

Related Questions