CODEWITHSUNDEEP
javahttpclientexecutorservice
Juan
Juan

Reputation: 15

ExecutorService performing REST requests

I´m trying to send an undefined number of HTTP Rest requests using concurrecy. This is the basic code skeleton:

PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
connManager.setMaxTotal(maxConn);
connManager.setDefaultMaxPerRoute(maxConn);
    CloseableHttpClient httpclient = HttpClients.custom().setConnectionManager(connManager).build();

ExecutorService es = Executors.newFixedThreadPool(maxConn);

for (...){
   es.execute(new RequestThread(httpclient, ...other data for creating a request...));
}

es.shutdown();
es.awaitTermination(timeout, TimeUnit.SECONDS);
//Tested timeout = 30

httpclient.close();

And the basic code of RequestThread is:

    public class RequestThread implements Runnable {

public RequestThread(CloseableHttpClient httpclient, ..other neccesay data..){
   ...
}

    @Override
    public void run() {
	try {
	    processRequest(httpclient, ...);
                // In this method, each CloseableHttpResponse is consumed and closed properly.
	} catch (Exception e) {
	    e.printStackTrace();
	}
    }

And it throws this exception multiple times:

java.lang.IllegalStateException: Connection pool shut down
    at org.apache.http.util.Asserts.check(Asserts.java:34)
    at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:251)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:175)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71)
    at com.ericsson.http.DMHttpCommunicationManager.executeRequestMethod(DMHttpCommunicationManager.java:109)
    at com.ericsson.http.RequestThread.processRequest(RequestThread.java:39)
    at com.ericsson.http.RequestThread.run(RequestThread.java:25)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

It seems the main thread is being closed too soon and causes the created runnables to throw these exceptions.

Is there any way to force ExecutorService to wait for the runnable to finish (before closing the HttpClient)? Should I use other strategy?

Thanks in advance.

Regards.

Upvotes: 0

Views: 3939

Answers (1)

Michael Cropper
Michael Cropper

Reputation: 1032

Try changing this line of code;

CloseableHttpClient httpclient = HttpClients.custom().setConnectionManager(connManager).build();

To;

CloseableHttpClient httpclient = HttpClients.custom().setConnectionManager(connManager).setConnectionManagerShared(true).build();

I was having a similar error when I came across this thread and this seemed to fix the issue for me. I know this is an old question, but adding thoughts for others for future reference.

I'm not 100% sure as to why this fix works as the documentation around this is pretty awful. It was a lot of trial and error with what I was testing to get to this solution. From what I can gather though, this fix works because it is then using a shared connection pool in the background, which means that connections remain open for use.

Upvotes: 1

Related Questions