David V.
David V.

Reputation: 43

Garbage collection acting weird

I'm just new to a project where they asked me to investigate why the servers (application) are acting weird. After a reboot they are blazingly fast (<150 ms response time), but when they run for about 2 days they become slow.

Memory and CPU go up and will not drop until a restart of the application.

So, they are running a Tomcat (hybris) server which has the following commandline flags: -XX:ConcGCThreads=1 -XX:G1HeapRegionSize=4194304 -XX:GCLogFileSize=786432 -XX:InitialHeapSize=12884901888 -XX:+ManagementServer -XX:MaxGCPauseMillis=200 -XX:MaxHeapSize=12884901888 -XX:NewRatio=4 -XX:NumberOfGCLogFiles=10 -XX:-OmitStackTraceInFastThrow -XX:ParallelGCThreads=4 -XX:+ParallelRefProcEnabled -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:ReservedCodeCacheSize=134217728 -XX:ThreadStackSize=1024 -XX:+UseCodeCacheFlushing -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseG1GC -XX:+UseGCLogFileRotation -XX:+UseTLAB

In the picture below you can see the CPU and Memory usage before and after the restart. The application server was under heavy load for a few hours already...

CPU & Memory usage

Heap & Eden Heap Usage

Old Gen Heap Usage

Garbage Collection CPU time

The application server itself is a 4-core with 16GB RAM.

Screenshots from a full run between 2 restarts:

CPU & Memory usage

Heap & Eden Heap usage

Old Gen Heap usage

Garbage Collection CPU time

Upvotes: 4

Views: 959

Answers (1)

P_M
P_M

Reputation: 2942

Your application have memory leak.

This is not a garbage collector (GC) issue, but the bug in your application. This means some of objects created, but was not cleaned up with GC, because reference links to them still exist in your application. You should investigate what objects are not cleaned and track down how they was created and where the references left.

As you mentioned TomCat I would first check Servlets (or controllers and services if you use Spring) for class property variables.

Upvotes: 4

Related Questions