CODEWITHSUNDEEP
javatomcat7inputstreamblockingresponse-time
Diptendu
Diptendu

Reputation: 21

Response time increases of Rest-API when there is high traffic

We are using Tomcat7 with Java 7. We are getting a throughput of around 500 req/sec. But with multiple instances in place and only 20 request is served by single instance.

The problem is as it goes over 20 requests on single instance response time adds up.

As for an example : -

***1. if 1st request takes : - 5ms (Millisecs)

  1. then 2nd request takes: - 10ms

  2. 3rd : - 15ms and so on.......***

I tried making it constant but it is not working.

What have we tried : -
1. Changing connector from BIO to NIO in Tomcat7.
2. Increasing maxThreads = 1000.
3. Increasing Cores from 2 cores to 10 cores.
4. Async API implementation.

None of the above worked.

What We want to achieve :-

  1. We would like to have constant response time of API.
  2. 100 req/sec in one single instance.

-----------------<>---------------<>--------------------------<>----<>--------------

Edit:-

Ok After a day of reading the code and trying different things. Found the main reason because of which performance is Low. This is due to InputStream read() function blocking nature. When java reads the o/p from socket . Socket returns the o/p in inputstream. Inputstream.read() is a blocking method in java (Link attached below). Still have no idea how to fix it.

Link : - http://javarevisited.blogspot.in/2012/02/what-is-blocking-methods-in-java-and.html#axzz4kWX7K7Yc

Have tried many methods of Java NIO but nothing is working so far.

Thanks in advance.

Upvotes: 1

Views: 2487

Answers (1)

galusben
galusben

Reputation: 6372

Look at the memory consumption, networking, and maybe start a profile. On what kind of hardware are you running? Make sure your java memory parameters are being passed correctly to tomcat. Enable garbage collection logs.

https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/geninfo/diagnos/memman.html

CATALINA_OPTS vs JAVA_OPTS - What is the difference?

Also, please describe the flow, there might be nothing bad with your tomcat, for example, if you are using a database that limits to X connections, then you are being trottled by the database.

What are the most common requests? What those request are doing?

Upvotes: 1

Related Questions