CODEWITHSUNDEEP
javajava-11java-http-client
Vladimir Petrakovich
Vladimir Petrakovich

Reputation: 4286

Close Java HTTP Client

Is there a way to close java.net.http.HttpClient to instantly release resources held by it?

Internally it holds a selector, a connection pool and an Executor (when default one is used). However it does not implement Closeable/AutoCloseable.

Upvotes: 32

Views: 23854

Answers (8)

KUL
KUL

Reputation: 491

Unfortunately, using the "ExecutorService" it is not possible to delete the streams "HttpClient-.*-SelectorManager". The only bad way to terminate streams is to find streams by name mask and force them to terminate. All streams associated with HttpClient will be terminated (java < 21):

    Thread.getAllStackTraces().keySet().forEach(t -> {
        if (t.getName().matches("HttpClient-.*-SelectorManager"))
            t.interrupt();
    });

Upvotes: 0

Yonas
Yonas

Reputation: 439

As per Java 21, HttpClient implements AutoCloseable. It also gains the methods shutdown(), shutdownNow() and awaitTermination(), which work similarly to the same-named methods on ExecutorService.

shutdownNow() should fill your needs.

Upvotes: 8

Stefan
Stefan

Reputation: 2395

This is a bit late, but I just want to highlight that the comment from Jacob G. (Dec 25, 2018) contained a solution that worked for me:

Creating the httpClient:

myExecutorService = Executors.newCachedThreadPool();
HttpClient myHttpClient = HttpClient.newBuilder() 
    .executor(executor) 
    ....
    .build();

Shutting down:

myExecutorService.shutDown();

Instead of having to wait for 90 seconds for the client to drop its connection, it happened "instantly".

Upvotes: 1

Will Xinwei Luo
Will Xinwei Luo

Reputation: 5

If it is just to gracefully close HttpClient at the end of the application lifecycle, System.exit(0) shall just work.

public static void main(String[] args) {
    ...
    System.exit(0);
}

I think it sends a interrupt signal to all threads in the JVM and HttpClient selmgr daemon does pick this up and shutdown itself.

final class HttpClientImpl extends HttpClient implements Trackable {
    ...
    // Main loop for this client's selector
    private final static class SelectorManager extends Thread {
        ...
        @Override
        public void run() {
            ...
            try {
                ...
                while (!Thread.currentThread().isInterrupted()) {...}
            } catch (Throwable e) {...}
            finally {
                ...
                shutdown();
            }

Upvotes: -2

kachanov
kachanov

Reputation: 2755

I had similar problem when I was redeploying a war file into Tomcat. War application had an HttpClient, that was running scheduled jobs issuing http requests and processing results.

I was seeing warnings from Tomcat when redeploying war file quite often on dev enrionment about hanging threads that may cause memory leaks. The stack trace was pointing to HttpClient threads. After several attempts I solved this problem in this way:

  1. HttpClient is created only when necessary to execute the job. It is not created as a field of a class or serivec, only as a local variable inside the scheduled method.

  2. HttpClient is created using builder and populated with a ThreadPool Executor, thus I keep the link to Executor and have control on it. ExecutorService executor = Executors.newSingleThreadExecutor(); HttpClient client = HttpClient.newBuilder().followRedirects(Redirect.ALWAYS).connectTimeout(Duration.ofSeconds(5)).executor(executor).build();

  3. When job is done in try-catch block, finally section has these two lines that: close the thread pool explicitly and set null to the httpClient local variable: executor.shutdownNow(); client = null; System.gc();

Notes, have short connection timeout to limit the time of execution. keep number of threads small. I use threadPool of 1 thread.

After all these changes warnings about memory leaks disappeared from Tomcat logs.

Upvotes: 11

Dmitry Zagorulkin
Dmitry Zagorulkin

Reputation: 8548

Obviously HttpClient is designed to be self managed. So it responsible to maintain connection pool, cache ttl by itself.

In HttpClientCode We could find the following code:

if (!owner.isReferenced()) {
                                Log.logTrace("{0}: {1}",
                                        getName(),
                                        "HttpClient no longer referenced. Exiting...");
                                return;
                            }

this is a graceful way to exit from SelectorManager loop and clean all resources.

 @Override
 public void run() {
            ...

            try {

                while (!Thread.currentThread().isInterrupted()) {

                    ...

                    if (!owner.isReferenced()) {
                        Log.logTrace("{0}: {1}",
                                getName(),
                                "HttpClient no longer referenced. Exiting...");
                        return;
                    }

                    ...
                }
            } catch (Throwable e) {
                ...
            } finally {
                ...
                shutdown();
            }
        }




    final boolean isReferenced() {
            HttpClient facade = facade();
            return facade != null || referenceCount() > 0;
        }

So when your HttpClient object will not be referenced, then it will clean all resources.

UPD: also you should tune your requests by passing timeouts

Upvotes: 2

Hadi
Hadi

Reputation: 49

In Java 11, each HttpClient spawn a daemon thread called selmgr which supposed to take care of in fly requests. This thread will be closed when there is no reference to the HttpClient in the code. However, in my experience, it is not reliable. Especially, when you use asynchronous methods with future timeouts.

Here is a piece of code I wrote using reflection to reliably close the HttpClient

static void shutDownHttpClient(HttpClient httpClient)
{
    ThreadPoolExecutor threadPoolExecutor = (ThreadPoolExecutor) httpClient.executor().get();
    threadPoolExecutor.shutdown();
    try {
        Field implField = httpClient.getClass().getDeclaredField("impl");
        implField.setAccessible(true);
        Object implObj = implField.get(httpClient);
        Field selmgrField = implObj.getClass().getDeclaredField("selmgr");
        selmgrField.setAccessible(true);
        Object selmgrObj = selmgrField.get(implObj);
        Method shutDownMethod = selmgrObj.getClass().getDeclaredMethod("shutdown");
        shutDownMethod.setAccessible(true);
        shutDownMethod.invoke(selmgrObj);
    }
    catch (Exception e) {
        System.out.println("exception " + e.getMessage());
        e.printStackTrace();
    }

}

As you can see, this is implementation-dependent and may not works with future Java versions. It is tested with Java 11 and Java 12.

Also, you need to add --add-opens java.net.http/jdk.internal.net.http=ALL-UNNAMED to your java command.

Upvotes: 2

Floegipoky
Floegipoky

Reputation: 3273

As you've noticed, java.net.http.HttpClient does not implement Closeable or AutoCloseable. So I can think of only 2 options, but neither of them are really bulletproof or even good:

You could eliminate every strong reference to the HttpClient that your program is holding and request a garbage collection. However there's a real risk that something beyond your direct control is holding onto it or one of its components. Any remaining strong references would prevent the referenced object, and any objects it holds a strong reference to, from being garbage collected. Nevertheless, this is arguably the more idiomatic option than the alternative.

I also found another option.

final class HttpClientImpl extends HttpClient implements Trackable {
    ...
    // Called from the SelectorManager thread, just before exiting.
    // Clears the HTTP/1.1 and HTTP/2 cache, ensuring that the connections
    // that may be still lingering there are properly closed (and their
    // possibly still opened SocketChannel released).
    private void stop() {
        // Clears HTTP/1.1 cache and close its connections
        connections.stop();
        // Clears HTTP/2 cache and close its connections.
        client2.stop();
    }
    ...
}

I wouldn't feel comfortable using this unless I had no other choice. Your reference is probably of type HttpClient, so you'd need to cast it to HttpClientImpl. It's bad to rely on the concrete implementation, which could change in future releases, rather than the HttpClient interface. The method is also private. There are ways around this but it's messy.

Upvotes: 3

Related Questions