Apache HttpComponents EntityUtils memory leak?

Question

I executed a 12-hour load test on my Java servlet-based server (Java 1.7 running under WebLogic 12 and Oracle Enterprise Linux). After test has finished, I observed that the memory consumption gradually went up from 500Mb in the beginning of the test to around 3.5Gb (and fluctuated there +/- 500Mb for several hours). 3 days later (server was doing nothing during these 3 days) - I examined memory again, and noticed that nothing out of these 3.5Gb was freed.

To make sure GC is executing, I mad explicit GC using jcmd:

• GC.run
• GC.run_finalization
• GC.run

Then, I made a heap snapshot and analyzed with YourKit analyzer (same results with jvisualvm as well).

I noticed that I have 2.5Gb of "Unreachable" objects still being there after 3 days of inactivity. A typical object's incoming references look like this (I'm having ~700K of such objects):

I examined Apache HttpComponents code (HttpCore version 4.3.2), and I noticed the following strange code in EntityUtils.toString (which I'm using in my server):

It looks to me that since reader object is not closed - it can create references, eventually causing the leaks I'm observing.

• Is this a leaky code of Apache HttpCore EntityUtils?
• Any other reason to have so many unreachable objects after 3 days and after GC.run?

Thank you.

Answer 1

Consider extending AsyncCharConsumer as shown here to produce a string with response content or using a HttpAsyncResponseConsumer if need a more sophisticated response processing.

EntityUtil methods are generally intended for entities backed by a blocking InputStream. And even then I would generally recommend consuming entity content directly from the content stream instead of converting it to a string.