Java memory leak - FileReaders, BufferedReaders and Stacks

Question

I'm not the worlds most experienced Java programmer, and I'm trying to create a tool which ingests data and stores it away, but I've got a memory leak (at least I think that's what's happening, either way I'm getting a java.lang.OutOfMemoryError: Java heap space).

Here's the offending code:

public static void main(String[] args)
{
    File inputDirectory = new File(args[0]);
    File[] files = inputDirectory.listFiles();
    for(int i = 0 ; i < files.length ; ++i)
    {
        DoSomethingWithTheFile(files[i]);
    }
}

public static void DoSomethingWithTheFile(File inputFile) throws VariousExceptions
{
    boolean result = false;
    if(inputFile.exists())
    {
        FileReader fr = new FileReader(inputFile);
        BufferedReader br = new BufferedReader(fr);
        Stack lines = new Stack();
        String line;
        while((line = br.readLine()) != null)
        {
            lines.push(line);
        }
        br.close();
        int numberOfRecords = lines.size();
        for(int i = 0 ; i < numberOfRecords ; ++i)
        {
            String theString = (String)lines.pop();
        }
        result = true;
    }
    return result;
}

I'm reading into a stack so that I know the number of lines in the file before I start processing, so that I can give the user some idea of how far into the file we are.

Obviously this is a toy example, and in the real version it does more than this, however I can replicate the issue with the above, stripped down version of the code. I'm currently trying to read 4 files, the first of which has just over half a million lines, the second just over a quarter of a million. The program bombs out whilst trying to read the third file.

Any ideas of what I'm doing wrong?

Thanks, Rik

Answer 1

You could increase the max memory allocate to your program with -Xmx option:

java -Xmx256m -jar yourApplication.jar

Besides, a better way to close the stream is to use try / catch / finally. The "finally" statement is always run, even if an exception is thrown. It will be something like:

boolean result = false;
if(inputFile.exists())
{
    FileReader fr = null;
    BufferedReader br = null;
    try{
        fr = new FileReader(inputFile);
        br = new BufferedReader(fr);
        Stack lines = new Stack();
        String line;
        while((line = br.readLine()) != null)
        {
            lines.push(line);
        }
        br.close();
        int numberOfRecords = lines.size();
        for(int i = 0 ; i < numberOfRecords ; ++i)
        {
            String theString = (String)lines.pop();
        }
        result = true;
    }
    finally{
        if( br != null ) br.close();
        if( fr != null ) fr.close();
    }
}
return result;

Finally, that's the "Stack lines" objet which is retaining a lot of memory. Perform your operations in the while(...) loop if you don't want to increase memory.

Answer 2

Have you tried increasing the maximum heap size?

java -Xms<initial heap size> -Xmx<maximum heap size>

Edited: Ow.. There already an answer