CODEWITHSUNDEEP
javaperformancetomcatservlets
Staros
Staros

Reputation: 3282

Managing Tomcat Requests per Servlet

I'm hoping to get some best practice guidance from the community.

Today I help manage a Java web application. We utilize Tomcat as the container and there are about 50 servlets in total. Most of these are extensions of the standard HttpServlet and haven't been updated to take advantage of any of the new asynchronous processing capabilities of the Servlet 3.X spec. These servlets power things like UI communication, client device communication, etc, etc.

The issue we're hoping to mitigate is a glut of requests through one servlet can starve out the others. This is two fold. First, a glut of requests can consume 100% of our system resources and leave the system unresponsive. We can throttle & tune these requests so they consume less resources, but this will often lead to a pile up in the connection pool. Either way we end up in a situation where the other servlets are unresponsive.

We're currently investigating solutions that would include utilizing Servlet 3.0's async functionality and thread pools to back the different areas of the application. First, this would allow us to better tune the application to accommodate the different types of request. Second, this would allow us to evaluate the type of request and prioritize it accordingly. We're always going to have a resource limit and at least this approach allows us to process the most important work.

A theoretical difficulty we're debating is that until we get everything converted to this new approach (which could be a while), the classic servlets and requests can still starve the application.

My specific questions for the SO community are...

  1. While we work towards this mechanism, is there any way for us to limit requests on a servlet by servlet basis to prevent saturation?
  2. Are there any resources out there on "best practices" this type of conversion?
  3. Are there any pitfalls with our desired state? If so, what advice do you have to mitigate them?

Finally, I do realize that at a large scale this is a byproduct of an single monolithic application that is handling lots of different types of requests. We're currently undertaking an effort to modularize the application and possibly get us to a point where we can distribute the application across different systems.

Thanks!

Upvotes: 3

Views: 368

Answers (2)

Marinos An
Marinos An

Reputation: 10808

To limit requests on a per-servlet basis, you could use a filter that only matches the specific servlet name(s), and allows only a specific number of requests to be served concurrently.

From servlet 2.5 spec:

Only one instance per <filter> declaration in the deployment descriptor is instantiated per JVM of the container

Inside your filter you can use a Semaphore to limit requests

ConcurrentRequestLimiter.java 

private final int MAX_CONCURRENT_REQUESTS = 5;
private final Semaphore limiter = new Semaphore(MAX_CONCURRENT_REQUESTS);
public void doFilter(ServletRequest request, ServletResponse response,
        FilterChain chain)
        throws IOException, ServletException {
    try {
        if (!limiter.tryAcquire(0, TimeUnit.SECONDS)) {
            PrintWriter out = response.getWriter();
            try {
                out.print("System busy. Please try again later..");
            } finally {
                out.close();
            }
        }else{                
            try{
                chain.doFilter(request, response);
            }finally{
                limiter.release();
            }

        }
    } catch (InterruptedException ex) {/*Should never happen*/throw new ServletException("System inconsistency");}
}

web.xml

<filter>
    <filter-name>ConcurrentRequestLimiter</filter-name>
    <filter-class>ConcurrentRequestLimiter</filter-class>
</filter>
<filter-mapping>
    <filter-name>ConcurrentRequestLimiter</filter-name>
    <url-pattern>/FilteredRequest</url-pattern>
</filter-mapping>
<servlet>
    <servlet-name>FilteredRequest</servlet-name>
    <servlet-class>FilteredRequest</servlet-class>
</servlet>
<servlet-mapping>
    <servlet-name>FilteredRequest</servlet-name>
    <url-pattern>/FilteredRequest</url-pattern>
</servlet-mapping>

Note: if you wish to use differrent thresholds for differrent servlets you can declare the filter multiple times with differrent alias and pass num-concurrent-requests as a config parameter to it.

Upvotes: 0

Eug&#232;ne Adell
Eug&#232;ne Adell

Reputation: 3174

If I can suggest an not so off-topic answer, you can give different priorities to different Executors.

It needs to be tested to see if the benefits are real and good, but if you can afford to run the application twice (once by Executor) this could be a temporary solution. The guilty servlet would be handled by the lowest priority Executor, while all the rest would be handled by the highest priority.

I see different ways to implement this. By changing the web.xml to hide the servlet in application 1 and forward to application 2, or by implementing a filter to intercept the calls to this guilty servlet and then sending it to the 2nd app for example.

Upvotes: 1

Related Questions