markus
markus

Reputation: 57

How to restrict the Java VM overall memory consumption?

I am running a Java application on a Linux-Cluster with SLURM as resource manager. To run my application I have to specify for SLURM the amount of memory I will need. SLURM will run my application in a kind of VM with the specified amount of memory. To tell my java application how much memory it can use I use the "-Xmx##g" parameter. I choose it 1GB less than I have requested from SLURM.

My problem is that I am exceeding the amount of memory I have chosen on SLURM and it terminates my application. It seems that the JVM uses about 1GB of memory, probably for things like GC or so.

Is there a possibility to restrict the size of the JVM or at least to tame it.

Cheers, Markus

Upvotes: 2

Views: 883

Answers (1)

Peter Lawrey
Peter Lawrey

Reputation: 533510

The maximum heap setting only limited the maximum heap. There are other memory regions which you have not limited such as

  • thread stacks
  • perm gen
  • shared libraries
  • native memory used by libraries
  • direct memory
  • memory mapped files.

If you want to limit the over all memory usage you need to be clear about whether you are limiting virtual memory or resident memory. Often monitoring tools make the mistake of monitoring virtual memory which shows a surprising lack of understanding of how applications work, or even why you monitor an application in the first place.

You want to monitor resident memory usage which means you need to know how much memory your application uses over time apart from the heap, then work out how much heap you can have plus some margin for error.

. To tell my java application how much memory it can use I use the "-Xmx##g" parameter. I choose it 1GB less than I have requested from SLURM.

At a guess I would start with 1/2 GB with -Xmx512m and see what is the peak resident memory and increase it if you find there is always a few hundred MB head room.

BTW 1 GB of memory doesn't cost that much these days (as little as $5). Your time could be worth much more than the resources you are trying to save.

Upvotes: 3

Related Questions