CODEWITHSUNDEEP
javafilememoryout-of-memory
Edward
Edward

Reputation: 557

Java - byte buffer Out Of Memory

Sort of new to Java but I've been able to learn quite a bit quite quickly. There are still many methods that elude me though. I'm writing a Java program that is to run through a bunch of files and test them thoroughly to see if they are a valid level pack file. Thanks to Matt Olenik and his GobFile class (which was only meant to extract files) I was able to figure out a good strategy to get to the important parts of the level pack and their individual map files to determine various details quickly. However, testing it on 37 files (5 of which aren't level packs), it crashes after testing 35 files. The 35th file is a valid level pack. The error it gives is:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

     Iterator iter = itemList.keySet().iterator();
     numJKLs = 0;
     while (iter.hasNext()) {
        String nextFile = (String) iter.next();
        if (nextFile.startsWith("jkl\\")) {
           System.out.println(((ItemInfo) itemList.get(nextFile)).length);
-->        byte[] buffer = new byte[((ItemInfo) itemList.get(nextFile)).length];
           gobRaf.seek(((ItemInfo) itemList.get(nextFile)).offset);
           gobRaf.read(buffer, 0,
                 ((ItemInfo) itemList.get(nextFile)).length);
           String[] content = new String(buffer).toLowerCase().split(
                 "\n");
           levels.add(numJKLs, new JKLFile(theFile, nextFile, content));
           gametype = levels.get(numJKLs).getFileType();
           progress.setProgText(nextFile);
           buffer = null;
           content = null;
           numJKLs++;
        }
     }

Arrow shows where the error is marked. The ´JKLFile´ class reads the content array for these important parts, but should (in theory) dispose of it when done. Like here, I set content = null; in JKLFile just to make sure it is gone. If you are wondering how big the map file it stopped on is, well, it managed to pass a 17 Mb map file, but this one was only 5 Mb.

As you can see these JKLFile objects are kept in this object (GobFile.java) for easy access later on, and the GobFile objects are kept in another class for later access, until I change directory (not implemented yet). But these objects shouldn't be too packed with stuff. Just a list of file names and various details.

Any suggestions on how I can find out where the memory is going or what objects are using the most resources? It would be nice to keep the details in memory rather than having to load the file again (up to 2 seconds) when I click on them from a list.

/Edward

Upvotes: 0

Views: 2049

Answers (1)

indivisible
indivisible

Reputation: 5012

Right now you are creating a new buffer for every loop. Depending on your garbage collection some of these may not get recycled/freed from memory and you are then running out. You can call the garbage collector using System.gc() but there is no guarantee that it actually runs at any given time.

You could increase the memory allocated to the JVM on the command line when calling your program with the -Xmx and -Xms arguments. Also if you are not on 32bit windows ensure you are using a 64bit JVM as the 32bit has memory limitations that the former does not. Run java -version from the command line to see the current system JVM.

You can move the declaration for your byte[] buffer outside your loop (above your while) so that it gets cleared and reused on each iteration.

Iterator iter = itemList.keySet().iterator();
numJKLs = 0;
byte[] buffer;
while (iter.hasNext()) {
    ...
    buffer = new byte[] ...
    ...
}

How big are the files you are handling? Do you really need to add every one of them to your levels variable? Could you make do with some file info or stats instead? How much RAM has your JVM currently got allocated?

To look at the current memory usage at any given time you will need to debug your program as it runs. Look up how to do that for your IDE of choice and you will be able to follow any Object's current state through the program's lifespan.

Upvotes: 1

Related Questions