CODEWITHSUNDEEP
javajvmmemory-managementpage-fault
Enas Yunis
Enas Yunis

Reputation: 1

Java JVM Keep Memory Allocation Against Operating System

I am running 1.6.0_25 64bit on windows 7 64bit.

I am trying to have the JVM run with maximum memory because my application is very memory intensive... but sadly... the memory allocation does not build up and there is a lot of Page Faults from windows to keep bringing the next set of virtual memory in.

I am running java -Xms2G -Xmx3G Test

The following code is my attempt at bringing a whole file in... such that i do not run into page faults for reading in.

File f = new File("veryLARGEfile.txt");
FileInputStream in = new FileInputStream(f);
int i = (int) f.length();
System.out.println("Bytes in file: " + i);

byte[] file = new byte[i];
i = in.read(file);
System.out.println("Bytes read: " + i);

doing this method i can see that in windows task manager the system reaches 2G worth of memory while it is reading the file... but once it is done reading it... the memory falls back down again!!!!

This is a major problem... i need the whole byte array to stay in active memory.

Thank you, ey

I have modified the code to use basic array types int[][] and float[][] to hold my data instead of keeping ArrayLists of an object containing int's and float's.

doing this, i find that my java memory does not get swapped (so, i guess heap memory is treated a bit differently from stack here) [oh, i did change all the code to be static typed as well - i know, very bad programming style]

the issue that i am running into now is how to handle my HashMap... all my attempts of trying to build a lookup table is failing with O(n^2) running time to build!!!

Upvotes: 0

Views: 1722

Answers (2)

Peter Lawrey
Peter Lawrey

Reputation: 533492

Java tries fairly hard to keep all its application in memory, or put another way, it swaps out as a last resort as it perform very badly if any of its memory is swapped to disk.

If your Java application is swapping to disk, its because your application uses too much memory for the system you have. If your application is memory intensive and you have to use this much memory I suggest sure your system has enough memory. BTW. You can buy a server with 8 GB for ~$500, and new 16 GB workstation for $1000. ;)

Upvotes: 0

Michael Borgwardt
Michael Borgwardt

Reputation: 346260

The decision what parts of a process's memory to keep in RAM and which to swap out is made by the OS, not by the JVM. Generally, the OS will keep memory that is accessed frequently in RAM - and if it isn't, why would you need it there?

There might be OS APIs that allow you to "pin" a certain piece of memory in RAM, but this functionality is not exposed by Java.

But for your requirements, you should look into having the file memory-mapped rather than reading it in explicitly. It will likely be much faster.

Upvotes: 1

Related Questions