CODEWITHSUNDEEP
node.jsamazon-web-servicesexpressgoogle-cloud-platformgoogle-cloud-tasks
Sanyam Jain
Sanyam Jain

Reputation: 3025

One Instance-One Request at a time App Engine Flexible

I am using

My Requirements

because of my unique requirement, I want 1 request to be handled by only 1 instance. when it gets free or the request gets timed-out, only then it should get a new request. I have managed to reject other requests while the instance is processing 1 request, but not able to figure out the appropriate automatic scaling settings.

Please suggest the best way to achieve this. Thanks in advance!

Upvotes: 3

Views: 656

Answers (2)

Jofre
Jofre

Reputation: 3898

You could set readiness checks on your app.

When an instance is handling a request, set the readiness check to return a non-ready status. 429 (too many requests) seems like a good option.

This should avoid traffic to that specific instance.

Once the request is finished, return a 200 from the readiness endpoint to signal that the instance is ready to accept a new request.

However, I'm not sure how will this work with auto-scaling options. Since the app will only scale up once the average CPU is over the threshold defined, if all instances are occupied but do not reach that threshold, the load balancer won't know where to route requests (no instances are ready), and it won't scale up.

You could play around a little bit with this idea and manual scaling, or by programatically changing min_instances (in automatic scaling) through the GAE admin API.

Be sure to always return a 200 for the liveness check, or the instance will be killed as it will be considered unhealthy.

Upvotes: 4

Averi Kitsch
Averi Kitsch

Reputation: 921

In your app.yaml try restricting the max_instances and max_concurrent_requests.

I also recommend looking into rate limiting your Cloud Tasks queue in order to reduce unnecessary attempts to send requests. Also you may want to increase your MIN_INTERVAL for retry attempts to spread out requests as well.

Your task queue will continue to process and send tasks by the rate you have set, so if your instance rejects the request it will go into a retry pattern. It seems like you're focused on the scaling of App Engine but your issue is with Cloud Tasks. You may want to schedule your tasks so they fire at the interval you want.

Upvotes: 5

Related Questions