CODEWITHSUNDEEP
javaspring-boottomcatmemoryjvm
PaulDaviesC
PaulDaviesC

Reputation: 1261

Accounting for Java memory consumption

We are running a Java spring boot application on AWS. The platform we use is Tomcat 8.5 with Java 8 running on 64bit Amazon Linux/3.3.6. The machines are 4GB machines. We run this Java application with JVM args -XMX and -XMS as 1536m. The problem we are facing is that these instances quite frequently goes in to warning state due to 90%+ memory usage. Now we are trying to account for memory usage process by process.

To start with we just ran the top command on these machines. Here is the part of output.

top - 11:38:13 up 4:39, 0 users, load average: 0.90, 0.84, 0.90Tasks: 101 total, 1 running, 
73 sleeping, 0 stopped, 0 zombieCpu(s): 31.8%us, 3.7%sy, 5.6%ni, 57.2%id, 0.3%wa, 0.0%hi, 1.5%si, 0.0%st
Mem: 3824468k total, 3717908k used, 106560k free, 57460k buffersSwap: 0k total, 0k used, 0k free, 300068k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2973 tomcat 20 0 5426m 2.2g 0 S 37.1 60.6 173:54.98 java

As you can see Java is taking 2.2GB of memory. We have given XMX as 1.5GB. Although, we are aware that by using XMX we are just restricting heap, we wanted to analyse where exactly this extra 0.7GB is going towards. Towards that end, we decided to use NewRelic. And here is the graph on non heap memory usage.

enter image description here

The total memory non heap memory usage we could see comes around ~200MB. So with this 200MB and 1.5GB heap memory, we expect the total memory to be consumed by Java to be 1.7GB. This 1.7GB figure is also confirmed from NewRelic graphs as below:

enter image description here

As I mentioned earlier, the top command is telling us the Java is taking 2.2GB of memory. However we could only account for 1.7GB using NewRelic. How can we reconcile this extra 0.5GB of memory?

Upvotes: 0

Views: 303

Answers (1)

Juraj Martinka
Juraj Martinka

Reputation: 4348

There's more than you see on the NewRelic's non-heap memory usage graph. E.g. there are also Thread stacks which can occupy up to 1MB per thread. There's a JVM feature called Native Memory Tracking that you can use to track some of the non-heap memory usage. There can still be native allocations that aren't tracked at all.

I suggest you look at these excellent resources from @apangin:

Upvotes: 2

Related Questions