CODEWITHSUNDEEP
springspring-bootjmeter
Robin Sun
Robin Sun

Reputation: 1620

Why the SpringBoot website refuses clients' connection after several minutes of the Jmeter load test begins?

It is a SpringBoot website and deployed in one Linux server. We use Jmeter to do the load test.

We mock 500 users to visit the webiste index page simultaneously. The index page is very simple html, no database connection,so it is a quite short connection.

After about 2 minutes, Jmeter starts to throw timeout exception as bleow

enter image description here

I guess this is because of website reaching its capacity and running out of connection.

I get one quesiton here, why does website reach its capacity 2 minutes later after Jemter starts. If its TCP connection capacity for this website is 1000, I guess it will reach 1000 very soon after the Jmeter starts, not 2 minutes.

Besides, I see many TCP connections are in TIME_WAIT status in Linux server. I guess this may be related with the connection timeout?

Edit: Someone thinks it is running of port. Someone thinks it is running out of connection. And someone thinks it is running out of processing thread(eg. What does this messge java.net.ConnectException/Connection timed out mean in log.jtl file of Jmeter?). I don't know which one is the exact reason...

Upvotes: 0

Views: 531

Answers (1)

Dmitri T
Dmitri T

Reputation: 168092

Most probably this is due to underlying Linux TCP/IP kernel stack configuration, as per Linux TCP/IP tuning for scalability article:

By default, a connection is supposed to stay in the TIME_WAIT state for twice the msl. Its purpose is to make sure any lost packets that arrive after a connection is closed do not confuse the TCP subsystem (the full details of this are beyond the scope of this article, but ask me if you’d like details). The default msl is 60 seconds, which puts the default TIME_WAIT timeout value at 2 minutes. Which means you’ll run out of available ports if you receive more than about 400 requests a second, or if we look back to how nginx does proxies, this actually translates to 200 requests per second. Not good for scaling.

SO double check timeouts along with maximum number of ports/sockets/files on the Linux server - my expectation is that the aforementioned parameters need to be tuned for high loads.

It's also a good practice to have monitoring of baseline OS health metrics in place (CPU, RAM, Network, Disk, swap usage, etc.). You can use i.e. JMeter PerfMon Plugin or JMeter SSHMon Listener for this.

Upvotes: 2

Related Questions