Reputation: 43
I'm just new to a project where they asked me to investigate why the servers (application) are acting weird. After a reboot they are blazingly fast (<150 ms response time), but when they run for about 2 days they become slow.
Memory and CPU go up and will not drop until a restart of the application.
So, they are running a Tomcat (hybris) server which has the following commandline flags: -XX:ConcGCThreads=1 -XX:G1HeapRegionSize=4194304 -XX:GCLogFileSize=786432 -XX:InitialHeapSize=12884901888 -XX:+ManagementServer -XX:MaxGCPauseMillis=200 -XX:MaxHeapSize=12884901888 -XX:NewRatio=4 -XX:NumberOfGCLogFiles=10 -XX:-OmitStackTraceInFastThrow -XX:ParallelGCThreads=4 -XX:+ParallelRefProcEnabled -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:ReservedCodeCacheSize=134217728 -XX:ThreadStackSize=1024 -XX:+UseCodeCacheFlushing -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseG1GC -XX:+UseGCLogFileRotation -XX:+UseTLAB
In the picture below you can see the CPU and Memory usage before and after the restart. The application server was under heavy load for a few hours already...
The application server itself is a 4-core with 16GB RAM.
Screenshots from a full run between 2 restarts:
Upvotes: 4
Views: 959
Reputation: 2942
Your application have memory leak.
This is not a garbage collector (GC) issue, but the bug in your application. This means some of objects created, but was not cleaned up with GC, because reference links to them still exist in your application. You should investigate what objects are not cleaned and track down how they was created and where the references left.
As you mentioned TomCat I would first check Servlets (or controllers and services if you use Spring) for class property variables.
Upvotes: 4