CODEWITHSUNDEEP
javamemorymemory-managementjvm
rufushuang
rufushuang

Reputation: 300

How to make JVM use the max (all remain) memory of a server

I have an DFS algorithm java console application, that runs faster when more memory is provided. Just a DFS algorithm application, with neither I/O nor other outer-JVM resource usage. It consumes only CPU and memory. The application can run with an 1GB memory, but run much more faster with 2 GB memory. More memory provided, faster the application can run. I haven't touch the speed limit as 12GB of memory provided. So I must use all remain memory of a server to speed it up. And the application need not parallel, one request only at one time.

And I need to install the application on different server with different memory size.

Is there a way to let JVM use the all remain memory of the server?

-XX:MaxRAMFraction=1

MaxRAMFraction is not EVERY server good, some server will result in start JVM failure as location memory failure, some works good.

Use an wrapper application get system remain memory, and minus some memory usage other than Xmx, then start the real application with same Xms and Xms. The method will also result in JVM memory allocation error. Because the code below returns much more than memory we can use, not just a minus of Xss256m or some more non-heap JVM memory.

com.sun.management.OperatingSystemMXBean mbean = (com.sun.management.OperatingSystemMXBean)
    ManagementFactory.getOperatingSystemMXBean();
long size = mbean.getFreePhysicalMemorySize();

So is there a good way to let JVM use all remain memory of a server?

Upvotes: 2

Views: 1232

Answers (1)

Peter Lawrey
Peter Lawrey

Reputation: 533492

For large regions of memory I use off heap and this reduces overhead on the GC, one of the benefitis is that is can be any size at runtime and even larger than main memory if you do it carefully. You can use direct ByteBuffers but I use a library I wrote which extends the ByteBuffer functionality (>> 2 GB and thread safe) Chronicle Bytes The largest any one uses this is ~100 TB of virtual memory mapped to disk.

We have two data structures on top of Chronicle Bytes, a key-value store Chronicle Map and a queue/journal Chronicle Queue. This can make storing data off heap easier with a higher level interface.

The way the heap works, it has to reserve the maximum heap size on start up as a single continuous block of virtual memory. In particular, the GC assumes random access to this memory on a clean up which means if you are have slightly over utilised your memory, possibly because a process started after yours and some of the heap is swapped out you will see a dramatic fall in performance for your whole machine. Windows tends to start swapping your GUI meaning you can't get back control without a power cycle. Linux isn't as bad, but you will want to kill your process at this point. This makes tuning it size to use all memory very hard if the usage of your machine changes.

By using virtual memory by comparison, the GC doesn't touch it so unused portions have little impact. You can have areas of virtual memory many times main memory but only your current working set matters, and this is a size entirely in your control at runtime. Note: on Linux you can have virtual memory sizes 1000x your free disk space, but use with care, if you run out by touching too many pages your program will crash.

Upvotes: 2

Related Questions