How to efficiently read a text file with delimited values in Java?

Question

I have a text file that looks something like this:

2, 20, 0, 9, 7, 17, ... (100 columns)
1, 3, 22, 11, 0, 0, ...
...
(100 rows)

Conditions:

1. I want to read each value as an int into a variable of type int[][]
2. It's not a csv file (there are other values in the file as well)
3. This is for Android development - no java libraries
4. I want it to be as efficient as possible - preferably not to use String at all

What i thought of, but don't really like:

while ((line = br.readLine()) != null) {
   String[] values = line.split(','); // this heavily uses String
   parseInts(values);
}

and:

while ((r = fileInput.read()) != -1) {
   char c = (char) r;
   if (c != ',' && c != '\n') {
      doSomething(c); // this is problematic when reading 2 digit numbers
   }
}

Keep in mind that i want a way to efficiently do this. Any help is appreciated. Thanks

Check for my complete solution below.

Answer 1

Using Andrei Ciobanu's answer, i used StreamTokenizer and works perfectly, exactly what i wanted, thanks!

StreamTokenizer st = new StreamTokenizer(reader);
st.parseNumbers();
st.eolIsSignificant(true);
st.whitespaceChars(',', ',');

tilePattern = new int[rowCount][columnCount];
int row = 0;
int column = 0;

boolean eof = false;
do {
    int token = st.nextToken();
    switch (token) {
        case StreamTokenizer.TT_EOF:
            eof = true;
            break;
        case StreamTokenizer.TT_EOL:
            row++;
            column = 0;
            break;
        case StreamTokenizer.TT_NUMBER:
            tilePattern[row][column] = (int) st.nval;
            column++;
            break;
    }
} while (!eof);

Answer 2

You can use parseNumbers() and set , and new-line as white characters http://docs.oracle.com/javase/7/docs/api/java/io/StreamTokenizer.html or http://developer.android.com/reference/java/io/StreamTokenizer.html

Answer 3

Here is an efficient code snippet for you, missing only an alternative parseInt implementation for the char buffer and some adjustments...:

public static void main(String args[]) throws IOException {
    //data source
    StringReader reader = new StringReader(
            "2, 20, 0, 9, 7, 17" + "\n" +
                    "1, 3, 22, 11, 0, 0"); //todo change to the actual data source

    //result
    int[][] values = new int[2][6]; //todo increase size to new int[100][100]
    int x = 0;
    int y = 0;

    //buffer
    char[] charBuffer = new char[3]; //todo check if increased size is necessary
    int ci = 0;

    //parse loop
    while (true) {
        final int c = reader.read();
        switch (c) {
            case ' ':
                //ignore
                continue;
            case ',':
            case '\n':
            case -1:
                //parse number
                values[y][x] = parseInt(charBuffer, 0, ci);
                ci = 0;
                break;
        }
        if (c == -1) {
            break;
        }
        switch (c) {

            case ',':
                //next column
                x++;
                break;
            case '\n':
                //next line
                x = 0;
                y++;
                break;
            default:
                //store digit
                charBuffer[ci++] = (char) c;
                break;
        }
    }

    System.out.println(Arrays.deepToString(values));
}

public static int parseInt(char[] charBuffer, int offset, int length) {
    //todo implement a faster performing parse method for a ``char[]``
    return Integer.parseInt(new String(charBuffer, offset, length));
}

Answer 4

For a non-single digit number, we know that we can identify the number using the commas.

So read the number untill you get a break.

How do you a character into a number.

consider 123

number char    num * 10 + char
   0    1          0 * 10 + 1 = 1
   1    2          1 * 10 + 2 = 12
   2    3          12 * 10 + 3 = 123 

So here's the program

List<Integer> = new ArrayList();
int number = 0;
while ((r = fileInput.read()) != -1) {
   char c = (char) r;
   if (c != ',' && c != '\n') {
       number = number * 10 + (int)c - 48;
   }
   else
      {
         list.add(number);
         number = 0; 
     }
}

Using a Scanner Using a Scanner, setting a delimiter makes you read easily without taking up any extra space.

List<Integer> list = new ArrayList();
File file = new File("10_Random.txt");

try {

    Scanner sc = new Scanner(file);
    sc.useDelimiter(",");
    while (sc.hasNextLine()) {
        list.add(sc.nextInt());
    }
    sc.close();
} 
catch (FileNotFoundException e) {
    e.printStackTrace();
}