CODEWITHSUNDEEP
javamultithreadingapache-httpclient-4.xjava.util.concurrent
Steve Cohen
Steve Cohen

Reputation: 4859

Apache HTTP Client: build simulator using multithreaded environment

I am building a standalone java application to generate load on a system, simulating real world conditions.

The application is multithreaded, using the concurrent framework to generate lots of pooled threads, each of which runs "sessions". When the session is complete, the runnable ends and the thread is returned to the scheduler pool. Each "session" consists of the following:

approximately 2 minutes are taken up all together per session

Each thread created by the concurrent pool maintains a connection (an HTTPClient) which is reused by subsequent sessions. After each request a call to HttpRequest.releaseConnection() is made, as is recommended.

It all works pretty well. But perhaps TOO well.

While keeping the connections open and releasing them gives optimum performance, since I am building a simulator I really DON'T want optimal performance. I want to simulate suboptimal performance. I want the server to have to go through connection establishment on each session.
I want to create the connection (embedded in an HTTP client) at the beginning of each session and close it at the end of the session.

To accomplish this, I simply close the HttpClient at the end of the session and set its variable to null (both in a finally clause). When an application session thread starts a new session, if the client is null, it builds a new one using the HttpClientBuilder. However, when I do that, I get all sorts of Connection pool errors that foul up the simulation.

Is there a RIGHT way to make connections suboptimally, as described above, with Apache HttpClient? Kind of a crazy question, but a real one.

Upvotes: 1

Views: 727

Answers (2)

vanOekel
vanOekel

Reputation: 6548

You can toggle re-use of connections within the pool using the builder's setConnectionReuseStrategy. The DefaultConnectionReuseStrategy will re-use connections whenever possible, but the NoConnectionReuseStrategy will close all connections returned to the pool.

I used these connection re-use strategies in reverse: in production no re-use was set (to ensure proper load-balancing - every new connection is directed to a healthy server), but during testing I had to switch back to the default re-use strategy since the test was creating so many connections that the test-machine quickly ran out of ports to use (after a local port is used the OS keeps the port in a waiting/cooldown room, part of the TCP protocol). The good thing is that the test-code only defers from production code for this one setting of the connection re-use strategy.

Note that the combination of a connection pool and no re-use of connections still has it purpose: the pool will prevent more than it's maximum allowed size of open connections. E.g. if the application decides it wants to open 100 connections at the same time, and the pool has a maximum size of 30, the pool will let the other requests for the other 70 connections wait until connections are returned. This is a good way to make clients behave nice and prevent them from overloading the server.

Upvotes: 1

Mike Laren
Mike Laren

Reputation: 8188

I ran into this problem too, just a few months ago. When you call HttpRequest.releaseConnection() the connection is pooled and doesn't get closed immediately. When you started creating new HttpClients you also stopped making use of these connections pools and starting creating new TCP connections every time that you needed a request and that's what caused the problem. When if you have multiple threads creating and releasing connections in a short period of time then Java (and even the underlying OS layer) start throwing errors which, based on your comments, is what I assume that it's happening to you.

This is a short list of things that I tried at the time and seemed to reduce the rate of occurrence of this sort of errors:

  1. Set the TCP connection timeout values to low values (e.g. 5 secs). I was running my code using Java 1.7.0 on OSX and this made things better. Later on I stopped using HttpClient and switched to JerseyClient and having low timeouts made a positive difference too.
  2. If you're on OSX/Linux, use the ulimit command to bump up the number of maximum number file descriptors. That doesn't make connections close any faster but seems to let you keep more connections open and therefore delay the problem.
  3. Instead of having a single process with multiple processes (e.g. 1 process with 500 threads), try spawning multiple child processes with less threads (e.g. 5 processes with 100 threads each). I noticed that killing subprecesses helps with closing connections.

At the time I also ended up having to distribute the test agents over multiple hosts and use Grinder to synchronize them. Depending on how many concurrent connections you're trying to test your server with, but this might be the best way forward.

Upvotes: 1

Related Questions