CODEWITHSUNDEEP
javastringmemorymemory-managementjvm
Rahul Kumar
Rahul Kumar

Reputation: 2831

Memory concern for dealing with large String

In my application, I have written a REST call which would be used by developers and support persons only, where it loads data of n size from the logs file and return it as text/plain, where n is a configurable number which cannot be more than fixed size.

    RandomAccessFile file = new RandomAccessFile(logFinalPath, "r");
    //chunk default is 100000 max is 500000
    chunk = chunk <=0l?100000:(chunk>500000?500000:chunk);
    long fileSize = file.length();
    //start position is either chunk size from the end or when the file is smaller then from the beginning of file
    long pos = fileSize>chunk?file.length()-chunk:0;    
    int maxSize = (int) (fileSize>chunk?chunk:fileSize);
    file.seek(pos);
    byte[] bytes = new byte[maxSize];
    file.read(bytes);
    file.close();
    logger.info("fileSize : "+fileSize);
    return new String(bytes);

Now here is my question, we know that String are immutable and any new String created goes into String pool never to be cleaned till JVM is on and running. So in that case every time this REST call is made would not it be a hit on memory in the sense that it would keep loading large text into String pool?

If yes, what are the alternatives, I can't pass the byte array as that would be unreadable. How can it be better?

Update : Please note that this question is not about new constructor of String or literal representation, It is about how it can be optimized to avoid the large String being stored into string pool with or without a reference from heap object.

Upvotes: 1

Views: 765

Answers (1)

Peter Lawrey
Peter Lawrey

Reputation: 533432

we know that String are immutable

yes.

and any new String created goes into String pool

no, unless you call String.intern()

never to be cleaned till JVM is on and running.

String literals can be cleaned up from Java 6 as String literals were moved to the heap.

So in that case every time this REST call is made would not it be a hit on memory in the sense that it would keep loading large text into String pool?

Large objects are added into the tenured space and to clean those up you need a major collection which you generally you want to avoid.

There is a small performance hit, but not as big as a say converting a byte[] into a String only to convert it back again into bytes later. You could save a lot of work (and memory) by keeping the cached data as a byte[]

I would be tempted to memory map your source file on start up (this uses almost no heap no matter how big the file is) and copy from the source file directory to the output of the request as needed and avoid taking an on heap copy as much as possible.

Upvotes: 1

Related Questions