Tomcat via Apache Server going down after too many connections

Question

I have an Apache (2.4) Server that serves content through the AJP connector on a Tomcat 7 Server.

One of my clients manages to kill the tomcat instance after running too many concurrent connections to a JSP JSON Api service. (Apache still works, but tomcat falls over. Restarting Tomcat brings it back up) there are no errors in tomcats logs.

I would like to protect the site from falling over like that, but I am not sure what configurations to change.

I do not want to limit the number of concurrent connections as there are legitimate use cases for that,

My Tomcat memory settings are :
Initial Memory pool : 1280MB
Maximum memory pool : 2560MB
which I assumed was plenty.

It might be worth mentioning that the API service relies on multiple, possibly heavy MySQL connections.

Any advice would be most appreciated.

Answer 1

If you are talking about scaling, you need to do the horizontal scaling her with multiple tomcat servers.

If you cannot limit user connections & still want the app to run smooth, then you need to scale. Architectural change to microservices is an option but may not be possible always for a production solution.

The best to think about is running multiple tomcats sharing the load. There are various ways to do this. With your tech stack, I feel the Apache 2 load balancer plugin in combination with Tomcat will do best.

Have an example here.

Now, with respect to server capacity, db connection capacity etc, you might also need to think about vertical scaling.

Answer 2

Why don't you'd slowly switch your most used/ important application features to microservices architecture and dockerize your tomcat servers to be able to manage multiple instances of your application. This will hopefully help your application to manage multiple connections without impacting the overall performance of the servers at the job.