CODEWITHSUNDEEP
javaapache-httpclient-4.x
newmanne
newmanne

Reputation: 2089

Apache Httpclient abort requests causing error

I have a program that needs to solve several problems. For each problem, it can try several different algorithms in parallel. As soon as one algorithm solves the problem, I would like the others to terminate promptly. Some algorithms are accessed via HTTP requests: you make a POST with the problem to a server, block, and the server returns an answer. If another thread finds the answer before the server returns, I would like that thread to cancel the http request.

I was initially using Spring's RestTemplate, but I wasn't able to find this functionality. Eventually I stumbled upon Apache's httpclient (I am using org.apache.httpcomponents:httpclient:4.4.1). The Apache Httpclient says that it can abort requests, so I used it as follows:

    // fields
        private final Lock lock;
        private final AtomicBoolean activeSolve;
        private HttpPost post;
        private final CloseableHttpClient httpClient = HttpClients.createDefault();


public String makePost() {
    post = new HttpPost("some url");
    final StringEntity stringEntity = new StringEntity("some json string", ContentType.APPLICATION_JSON);
    post.setEntity(stringEntity);
    try {
        lock.lock();
        activeSolve.set(true);
        lock.unlock();
        try (final CloseableHttpResponse httpResponse = httpClient.execute(post)) {
            final String response = EntityUtils.toString(httpResponse.getEntity());
            return response;
        }
    } catch (IOException e) {
        if (post.isAborted()) {
            log.trace("Web request was aborted");
            return "aborted";
        } else {
            throw new RuntimeException("Could not contact server", e);
        }
    } finally {
        lock.lock();
        activeSolve.set(false);
        lock.unlock();
    }
}

and I have code that interrupts from another thread

   public void interrupt() {
        lock.lock();
        if (activeSolve.get()) {
            post.abort();
        }
        lock.unlock();
    }

This tends to work, but over the course of ~1000 problems (I've found it hard to reproduce, but if I run long enough I eventually see it), I'll get the following stacktrace:

java.lang.IllegalStateException: Connection is not open
at org.apache.http.util.Asserts.check(Asserts.java:34)
at org.apache.http.impl.BHttpConnectionBase.ensureOpen(BHttpConnectionBase.java:132)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseEntity(DefaultBHttpClientConnection.java:177)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseEntity(CPoolProxy.java:172)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:274)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at ... my classes

If I comment out the interrupt code and disable interrupting, I stop seeing this error. But as far as I can see, I'm using the library correctly.

If someone is familiar with this library, can you let me know how I am using it incorrectly or what sequence of events might trigger that error? And otherwise, if there is another library that cancels http requests that you've found reliable, I'm not bound to this apache library.

Upvotes: 1

Views: 5547

Answers (2)

user666
user666

Reputation: 1172

  • If you change your approach a bit and spawn your threads from withing a Thread pool and as soon as you get a response you can shut down the pool .
  • If I am not wrong in the above method I see that instance of HttpClient is shared ..try moving that within the critical section (method) and see then that interrupt thats causing above issues is gone .
  • the use of atomic variables is great in interrupt method.

Upvotes: 0

user1675642
user1675642

Reputation: 777

I'm not sure if this is the route of the problem, but HttpPost is marked as not thread safe (see the javadoc). This could well cause you issues, and might explain what you see here.

As it's not thread safe it's hard to know exactly how it will fail, and this could even depend on the JVM implementation.

Upvotes: 1

Related Questions