CODEWITHSUNDEEP
tomcatcrashloadtimeout
Ehsan Khodarahmi
Ehsan Khodarahmi

Reputation: 4922

Tomcat fails on heavy load

I'm running a heavy db-based GWT application on a debian VPS using tomcat server 7 (& JRE 1.6). My app contains a lot of java servlets which communicate with MySQL5 database via a tomcat connection pool (without connection pooling mysql will crash within less than 3 minute!)
My app works nice while there is no heavy load on tomcat server, but when number of online users & their requests increase, tomcat server fails with no useful log or error message(I just get a connection timeout error when I want to access any of web applications running on tomcat) & this problem exists till I reset my tomcat server. I know that I've NO memory limitation on my VPS nor any MySQL connection problem, so I really don't know that causes this situation :(
This is host tag in server.xml (I've many of these virtual hosts, but the host tag of all is similar)

<Host name="sub1.mydomain.com" appBase="/var/www/sites/gwt_app" >

    <Context path="" reloadable="true" docBase="myDocBase" 
     xmlValidation="false" xmlNamespaceAware="false" crossContext="true" >

    <Resource name="jdbc/mysql/db" auth="Container" type="javax.sql.DataSource"

    initialSize="3" maxActive="50" maxIdle="10" 
    maxWait="15000" removeAbandoned="true" removeAbandonedTimeout="120"
    validationQuery="select now();"

    username="user_1" password="pass" driverClassName="com.mysql.jdbc.Driver"
    url="jdbc:mysql://localhost:3306/db_1?useUnicode=true&amp;characterEncoding=UTF-8&amp;characterSetResults=UTF-8&amp;connectionCollation=UTF8_PERSIAN_CI&amp;noAccessToProcedureBodies=true"
    />
    </Context>

    <Valve className="org.apache.catalina.valves.AccessLogValve" directory="/var/www/sites/gwt_app_logs"  prefix="tomcat_access_" suffix=".log" pattern="common" resolveHosts="false"/>
</Host>

any idea?

Upvotes: 4

Views: 4814

Answers (4)

yacine ouah
yacine ouah

Reputation: 365

use tomcat clustering and apache network load blanc

Upvotes: 0

hcg
hcg

Reputation: 652

You may change CATALINA_OPTS parameter in /TOMCAT_DIR/bin/catalina.sh for Concurrent garbage collection 

CATALINA_OPTS="-XX:+UseConcMarkSweepGC -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -Xloggc:/TOMCAT_DIR/logs/gc.log"

Upvotes: 1

Tomasz Nurkiewicz
Tomasz Nurkiewicz

Reputation: 340733

You need to provide more details about your resources utilization. But based on your description you might be having an issue HTTP thread pool being exhausted. By default (with BIO connector) Tomcat can handle only 200 concurrent connections and AFAIK can keep that many connections "on hold" in the backlog queue.

This means that only 200 connections can be handled at the same time and another 100 will wait for the pool. 301st connection will be rejected or timeouted.

Here are pointers what to examine:

  • HTTP thread pool utilization
  • backlog queue size
  • garbage collection activity

Tomcat provides valuable metrics for that. You might also find my article about squeezing more concurrent connections from Tomcat useful.

Upvotes: 4

Pidster
Pidster

Reputation: 618

A recent bug in the BIO connector may be a factor here.

See: https://issues.apache.org/bugzilla/show_bug.cgi?id=53186

However, if your application also crashes in under 3 minutes with DB connection pooling, there is likely to be something wrong with your application or DB handling code.

Upvotes: 1

Related Questions