Reputation: 1165

Java heap memory capacity

While writing huge data to a file using java, how can one avoid java heap memory out of space error from occuring?..This is causing some failures for my program..

Regards,

magggi

Upvotes: 0

Answers (5)

dlongley

Reputation: 2096

From your comments it sounds like you're iterating over some objects that fill up a StringBuffer and, once you're done iterating, you flush the entire StringBuffer to a file. Instead of doing this, you can either write directly to an output stream on each iteration (recommended) or, on each iteration, write to your StringBuffer, flush it to the output stream, and then clear the StringBuffer for the next iteration (not preferred). This should lower your memory usage.

I don't recommend increasing the heap size -- it doesn't sound like the right solution to this problem.

Upvotes: 1

ILMTitan

Reputation: 11037

A couple of options

Increase Java Heap Size using the -Xmx JVM parameter. After -Xmx, write the number of megabytes your program needs to successfully run.
Use Streaming. Compute the next N bytes in the file and then write them using your FileOutputStream, throwing them away before computing the next N bytes. More code would be necessary to know if and how this would work for your program.

Edit: From what you are saying, it sounds like you could wrap the FileOutputStream in a BufferedOutputStream, drop the StringBuffer entirely and just write directly to the BufferedOutputStream.

Upvotes: 0

Colin Hebert

Reputation: 93197

Your problem is that you're using a single StringBuffer to handle all your content. You should try to flush your data more often. You will gain performance and memory.

Without the original code I can't help you more, but try to flush your buffer from time to time.

Not advised in your case

If you still want to use your StringBuffer and flush it only once, then you'll have to increase your heap space with the -Xmx***M option on the java command.

Resources :