CODEWITHSUNDEEP
javahttpclient
sherkal
sherkal

Reputation: 41

Multiple post with Httpclient 4.0.3 hanging randomly

let me explain the situation.

I have a servlet redirecting outgoing GET/POST to another project on another domain (some kind of proxy) whose job is to handle it and return some stuff (params and a gif). Im using HttpClient 4.0.3 to do this.

There are multiple GET/POST sent by my app on startup, so I setup once a ThreadSafeClientConnManager to handle multiple threads this way.

cm_params = new BasicHttpParams();
ConnManagerParams.setMaxTotalConnections(cm_params, 200);

ConnPerRouteBean connPerRoute = new ConnPerRouteBean();
HttpHost localhost = new HttpHost("localhost");
connPerRoute.setMaxForRoute(new HttpRoute(localhost), 50);

ConnManagerParams.setMaxConnectionsPerRoute(cm_params, connPerRoute);

SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(
        new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));

cm = new ThreadSafeClientConnManager(cm_params, schemeRegistry);

Then I create a new HttpClient with those params, which should be enough to handle a bunch of request at the same time. Of course, I do this on public void service() for every GET/POST but I use the same Httpclient object after I created it.

httpclient = new DefaultHttpClient(cm, cm_params);

After that I build up my POST and send it via execute, with all the requiered params and such triple verified.

    HttpPost httpPost = new HttpPost(target+tmpString); 

    httpPost.setHeader("Host", request.getHeader("host"));
    httpPost.setHeader("User-Agent", request.getHeader("user-agent"));
    httpPost.setHeader("Accept-Encoding", request.getHeader("accept-encoding"));
    httpPost.setHeader("Accept", request.getHeader("accept"));
    ..etc..

    UrlEncodedFormEntity urlEncodedFormEntity = new UrlEncodedFormEntity(params);
    urlEncodedFormEntity.setContentEncoding(HTTP.UTF_8);
    httpPost.setEntity(urlEncodedFormEntity);

    HttpResponse response = httpclient.execute(httpPost);

Finally I read the stream and handle the entities...

    OutputStream os = res.getOutputStream();
    InputStream is = response.getEntity().getContent();
    byte[] buf = new byte[1024];
    for(int n;(n=is.read(buf))!=-1;)
    {
        os.write(buf, 0, n);
    }
    // Make sure to close
    is.close();
    os.close();

    // Flush entities just in case
    EntityUtils.consume(urlEncodedFormEntity);
    EntityUtils.consume(response.getEntity());
    urlEncodedFormEntity.getContent().close();
    response.getEntity().getContent().close();

So my problem is, the code is working perfectly well when I load my page. There are 4 request sent (1 GET, 3 POST) correctly handled. Basicly returning some params and 1 small gif that I print on my page.

But as soon I start to Stress Test my app, I.E. loading the same page in 4-5 tabs, my app seems to hang randomly whenever I execute multiple POST at the same time. I thought I wouldn't have any problem even if I use the same Httpclient object since I declared my ThreadSafeClientConnManager correctly (I think?) so it should handle multiple thread.

Anyone know what I am doing wrong? If I load my tabs 1 by 1, it doesnt hang. Just when I refresh more than 1 tab at the same time.

Anyone have a clue? :S (sry english isnt my first language ^^;)

Upvotes: 4

Views: 4610

Answers (4)

Shikhar Srivastava
Shikhar Srivastava

Reputation: 23

I got stuck into the same error. What we noticed was that HttpClient doesn't close itself.

What I did was, I tried HttpClientUtils.closeQuietly() method on my uncleared HttpClient and HttpResponse.It helped my cause, and it stopped hanging anymore. You can also try and experiment with it.

Upvotes: 0

Ravi Raj
Ravi Raj

Reputation: 63

Recently i was trying to achieve this but i got some new problem like when first time i was sending request then it was sending without cookie but in second time httpContext was adding cookie automatically but this is the standard way to send but sometimes your requested server accept without cookie so for that i have used INTERCEPTOR. In second time it removes that auto added cookies after that it was working fine please use this httpClient interceptor use below code :

httpClient.addRequestInterceptor(new HttpRequestInterceptor() {

        @Override
        public void process(HttpRequest request, HttpContext context)
                throws HttpException, IOException {
            LOG.info("***************** Entered My Interceptor ****************************");
            Header[] headers = request.getAllHeaders();
            for (Header eachHeader : headers) {
                LOG.info("Headers -- Name: {}, Value: {} ",
                        eachHeader.getName(), eachHeader.getValue());
            }
            request.removeHeaders("Cookie");
        }
    });

Upvotes: 0

sherkal
sherkal

Reputation: 31

@Apache Fan, setting this :

ConnManagerParams.setMaxTotalConnections(cm_params, 200); connPerRoute.setDefaultMaxPerRoute(50);

Doubt I can run out of connection, I open 3-4 tab and it runs out... theres maybe 4 POST per tab so that doesnt add up :S

Upvotes: 3

Pushkar
Pushkar

Reputation: 7580

You may want to take a look at this from the HttpClient API -

ThreadSafeClientConnManager maintains a maximum limit of connection on a per route basis and in total. Per default this implementation will create no more than than 2 concurrent connections per given route and no more 20 connections in total. For many real-world applications these limits may prove too constraining, especially if they use HTTP as a transport protocol for their services. Connection limits, however, can be adjusted using HTTP parameters.

Maybe your application is running out of pooled connection and you need to increase the pool size.

Upvotes: 2

Related Questions