CODEWITHSUNDEEP
javaout-of-memorystringbuilderstreamreader
Persimmonium
Persimmonium

Reputation: 15789

how to create a reader out of a bunch of Strings without appending them in a StringBuilder first?

Imagine I have several Stings containing large ammount of data. I must pass the concatenation of them to a method in a library I use. I can use either a String or a Reader for that.

  1. Appending them and passing an String is out of question as I might run into an OOM. There is no option to change -Xmx anymore.
  2. So, is there a way to create a reader based on ALL my strings without having to create a single string and then to create a StringReader? This way I could avoid the OOM

ps: I already set the initial capacity of the StringBuilder for my first option, but this is no enough, so I am looking for a way to implement 2.

Upvotes: 1

Views: 335

Answers (4)

Steve Bennett
Steve Bennett

Reputation: 367

You could implement your own version of a reader which uses a base of arrays of strings and then loops through as the read method is called.

You should be able to subclass the Reader class, implementing your own versions of the read(char, int, int) and close() methods. The read(..) method could probably look something like the code below.

import java.io.IOException;
import java.util.Collection;
import java.io.*;

public class StringArrayReader extends Reader {
    private String[] strings;
    private int iString = 0, iCharInCurrentString = 0;

    public StringArrayReader(String[]strings) {
        this.strings = strings;
    }

    public StringArrayReader(Collection<String> strings) {
        this(strings.toArray(new String[strings.size()]));
    }



    @Override
    public int read(char[] buf, int off, int len) { 
        int iCurrentChar=0;
        while (iCurrentChar < len) {
            if (iCharInCurrentString < strings[iString].length()) {
                buf[iCurrentChar+off] = strings[iString].charAt(iCharInCurrentString);
            } else if (iString + 1 < strings.length) {
                iString++;
                iCharInCurrentString = 0;
                buf[iCurrentChar+off] = strings[iString].charAt(iCharInCurrentString);
            } else {   // current string is over and no more strings
                break;
            }
            iCurrentChar++;
            iCharInCurrentString++;
        }
        return iCurrentChar>0? iCurrentChar: -1;        
    }

    @Override
    public void close() throws IOException {
    }


    /* Demo */
    public static void main(String[] args) throws IOException {
        String s1 = "abcd";
        String s2 = "efgh";
        Reader r1 = new StringArrayReader(new String[]{s1,s2});
        int data = r1.read();
        while(data != -1){
            char dataChar = (char) data;
            System.out.println(dataChar);
            data = r1.read();
        }   
    }
}

Upvotes: 1

user949300
user949300

Reputation: 15729

I don't know of a built-in way or a 3rd party option, but it shouldn't be that hard to write your own Reader, taking a List (or array) of Strings. According to the Reader javadocs:

"The only methods that a subclass must implement are read(char[], int, int) and close()"

And close() should be easy. So you really only need to implement one method, read(char[], int, int). Keep indices of which String you are reading, and where you are in the String. The rest is left as an exercise for the Reader. :-) And you might want to implement read() for speed.

Upvotes: 2

Andy Thomas
Andy Thomas

Reputation: 86489

You could implement a custom Reader whose data is drawn from a list of your Strings. Your concrete subclass would be required to implement only two methods: read(char[], int, int) and close(). When your reader finishes consuming one String, move on to the next.

Upvotes: 1

James McLeod
James McLeod

Reputation: 2406

You could create an array of Strings and pass that. This will take up much less memory than copying all the Strings into one superstring.

You can also increase the amount of memory used by the JVM with the -Xmx option (e.g. -Xmx2g to allow up to 2 Gb of memory).

Upvotes: 0

Related Questions