CODEWITHSUNDEEP
amazon-web-servicesspring-bootjersey
vinit saha
vinit saha

Reputation: 57

restful API maximum limit : API Queue

I am developing a rest service using Spring boot. The rest service takes an input file and do some operation on it and return back the processed file.

I know that in spring boot we have configuration "server.tomcat.max-threads" which can be a maximum of 400.
My rest application will be deployed on a cluster.

I want to understand how I should be handling if the request is more than 400 for a case wherein my cluster has only one node.

Basically I wanted to understand what is the standard way for serving requests more than the "max-thread-per-node X N-nodes" in a cloud solution.

Upvotes: 0

Views: 258

Answers (1)

Mayank Raj
Mayank Raj

Reputation: 1624

Welcome to AWS and Cloud Computing in general. What you have described is the system elasticity which is made very easy and accessible in this ecosystem.

Have a look at AWS Auto Scaling. It is a service which will monitor your application and automatically scale out to meet the increasing demand and scale in to save costs when the demand is low.

You can set triggers for the same. For eg. If you know that your application load is a function of Memory usage, whenever memory usage hits 80% you can add nodes to the custer. read more about various scaling Policies here.

One such scaling metric is ALBRequestCountPerTarget. It will scale the number of nodes int he cluster to maintain the average request count per node(target) in the cluster. With some buffer, you can set this to 300 and achieve what you are looking for. Read more about this in the docs.

Upvotes: 1

Related Questions