CODEWITHSUNDEEP
javaheap-memory
Vasant
Vasant

Reputation: 301

How to manage Heap Space in Java

I'm trying to recursively traverse through my Drive to search some files. The code is working fine when there are limited folders/files but when I target my search to C drive where in I have lots of files it throws Out of Heap memory.

Exception in thread "Thread-4" java.lang.OutOfMemoryError: Java heap space

  1. Please suggest me some good memory management tricks especially when we do recursive calls.
  2. Or give me better approach to traverse through directories without recursion.

and I don't want to increase the maximum allowable heap space as it is just like postponing the issue for time being.

Code:

void iterateDirectory(String somedir) {


        File dir = new File(somedir);
        File[] files = dir.listFiles();
        if (files != null) {
            for (int id = 0; id < files.length; id++) {
                if (files[id].isDirectory() == false) 
                {
                   fsTree.add(files[id].toString()); // taking list of files
                } 
                else 
                {
                    iterateFilesInDirectory(files[id].getAbsolutePath());
                }
            }
        }
    }

Upvotes: 0

Views: 710

Answers (3)

ArjunShankar
ArjunShankar

Reputation: 23680

The culprit, as I see it is this line:

fsTree.add(files[id].toString()); // taking list of files

It appears that you add every single file to a global data structure (fsTree), and then search there.

My bet is:

A. It won't go away if you 'convert' your recursive function into an iterative one.

B. It will go away if, instead of appending to a global data structure and searching in the end, you do the search/matching locally, and only globally cache the matching hits:

void iterateDirectory (String somedir, String search_term) {

    File dir = new File(somedir);
    File[] files = dir.listFiles();
    if (files != null) {
        for (int id = 0; id < files.length; id++) {
            if (files[id].isDirectory() == false) 
            {
               if (/* files[id].isDirectory() MATCHES search_term */)
                 // add to list of matching files:
                 matching_hits.add(files[id].toString());
            } 
            else 
            {
                iterateFilesInDirectory(files[id].getAbsolutePath());
            }
        }
    }
}

Upvotes: 2

Makoto
Makoto

Reputation: 106508

There are limitations as to what recursion can do, namely with respect to stack/heap usage. Remember that whatever you can do recursively, you can do iteratively; rewrite your code to use an iterative solution instead.

Alternate solution: There is an interface in java.nio that can be used to recursively walk down a filesystem's structure. Take a look at this trail, and SimpleFileVisitor.

Upvotes: -1

NPE
NPE

Reputation: 500923

There are two possibilities:

  1. There is infinite recursion in your code (for example, because you're not processing . and/or .. correctly). If that's the case, you have to fix the code.
  2. Your code genuinely requires more heap space than what's available. You have two options:
    • reduce you process's memory requirements (a memory profiler could help you understand what's using all that heap space);
    • increase the heap size by specifying the -Xmx JVM option.

Upvotes: 1

Related Questions