why does BufferedReader fail to read the final line for an InputStream?

Question

*Note: To clarify, maybe this question wasn't quite clear on the InputStream being "alive." The connection to the telnet weather service (see below for link) is kept open. The goal is to get all lines coming from the server.*

Building from the sample Apache WeatherTelnet code, I'm using InputStream.read to output the server results (inspired by a javamex tutorial idiom) one char at a time, using the chars method:

thufir@dur:~$ 
thufir@dur:~$ java -jar NetBeansProjects/Teln/dist/Teln.jar 
------------------------------------------------------------------------------
*               Welcome to THE WEATHER UNDERGROUND telnet service!            *
------------------------------------------------------------------------------
*                                                                            *
*   National Weather Service information provided by Alden Electronics, Inc. *
*    and updated each minute as reports come in over our data feed.          *
*                                                                            *
*   **Note: If you cannot get past this opening screen, you must use a       *
*   different version of the "telnet" program--some of the ones for IBM      *
*   compatible PC's have a bug that prevents proper connection.              *
*                                                                            *
*           comments: jmasters@wunderground.com                              *
------------------------------------------------------------------------------

Press Return to continue:
^Cthufir@dur:~$ 

which is the desired output. However, reading the InputStream with a BufferedReader results in dropping the last line. (Or, at least, it's not printed to the console.) Bad output, using lines method:

thufir@dur:~$ 
thufir@dur:~$ java -jar NetBeansProjects/Teln/dist/Teln.jar 

------------------------------------------------------------------------------

*               Welcome to THE WEATHER UNDERGROUND telnet service!            *

------------------------------------------------------------------------------

*                                                                            *

*   National Weather Service information provided by Alden Electronics, Inc. *

*    and updated each minute as reports come in over our data feed.          *

*                                                                            *

*   **Note: If you cannot get past this opening screen, you must use a       *

*   different version of the "telnet" program--some of the ones for IBM      *

*   compatible PC's have a bug that prevents proper connection.              *

*                                                                            *

*           comments: jmasters@wunderground.com                              *

------------------------------------------------------------------------------



^Cthufir@dur:~$ 
thufir@dur:~$ 

StreamReadercode:

package teln;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;

public class StreamReader {

    private StreamReader() {
    }

    StreamReader(InputStream inputStream) throws IOException {
        lines(inputStream);
    }

    private void chars(InputStream inputStream) throws IOException {
        do {
            char ch = (char) inputStream.read();
            System.out.print(ch);
        } while (true);
    }

    private void lines(InputStream inputStream) throws IOException {
        BufferedReader br = new BufferedReader(new InputStreamReader(inputStream, "UTF-8"));
        String line = "";
        do {
            System.out.println(line);
        } while ((line = br.readLine()) != null);

        System.out.println(line);

    }
}

Presumably that last line is "null" somehow? Can the logic be altered so that the final line is printed from within lines just as it is from chars?

Answer 1

Ok, here goes the kludge:

package teln;

import static java.lang.System.out;
import java.io.IOException;
import java.io.InputStream;

public class StreamReader {

    private StreamReader() {
    }

    StreamReader(InputStream inputStream) throws IOException {
        printEachLine(inputStream);
    }

    private void printEachLine(InputStream inputStream) throws IOException {
        int foo = 0;
        char ch = 0;
        StringBuilder sb = new StringBuilder();
        out.println("lines..");
        boolean isEOL = false;  //err, need to set this or get rid of it
        do {
            foo = inputStream.read();
            ch = (char) foo;
            //out.print(foo);
            sb.append(ch);
            if ((ch == 10)) {
                out.print(sb);
                sb = new StringBuilder();
                foo = inputStream.read();
                ch = (char) foo;
                sb.append(ch);
                if (ch != 13) {
                    while ((255 > ch) && (ch >= 0)) {
                        sb = new StringBuilder();
                        foo = inputStream.read();
                        ch = (char) foo;
                        sb.append(ch);
                        out.print(sb);
                    }
                }
                sb.append(ch);
            }

        } while (!isEOL);

    }
}

this depends upon LF and CR following a specific pattern. LF, 10, is always followed by CR, 13, unless it's the first or last line. So, in that special case this code will look for the missing CR and identify that as needing special output.

However, it's a huge assumption and may be specific to this weather telnet service. Also, seems incredibly fragile.

For now, I'll go with this, but will keep looking for another answer.

Answer 2

You haven't explained where the data is coming from, but I suspect it's coming from some source which doesn't have a line terminator at the end and doesn't close the stream.

If the stream were closed - e.g. by killing the connection, if this is a client/server app - then you would see the final line. BufferedReader will definitely return the last line of data even if it doesn't end with a line terminator, but only when it knows it's got to the end of the stream.

Answer 3

A BufferedReader#readLine() reads the stream until it reaches either \n, \r, or the end of the stream, returning what it read up to then or null. You're not showing us something. Why are there new lines between lines in your output?

As for the null at the end of your output

private void lines(InputStream inputStream) throws IOException {
    BufferedReader br = new BufferedReader(new InputStreamReader(inputStream, "UTF-8"));
    String line = "";
    do {
        System.out.println(line); // will print an empty line to start
    } while ((line = br.readLine()) != null);

    System.out.println(line); // will print null

}

Your loop should be

while ((line = br.readLine()) != null) {
    System.out.println(line);
}

to avoid printing the first empty one. Also, don't print the line outside the while loop. It will always be null, obviously (you wouldn't be outside the while otherwise).

Your chars() method is also looping forever.

It's very possible your server isn't closing the stream and so the underlying read calls that the BufferedReader makes aren't returning -1 to indicate EOF. Are you killing the application yourself?

Answer 4

As Jean mentioned, this is a shortcoming of using the BufferReader if you know your input never ends with either the linefeed '\n' or carriage return '\r' character. After your while loop you may want to use the ready() method to test if there is indeed more text after the last line read, and use one of the read() methods to pull in that remaining text. Or, substitute another text reader for BufferReader all together.