CODEWITHSUNDEEP
javaprocessstreamstdoutunbuffered
R2-D2
R2-D2

Reputation: 1624

Reading other process' **unbuffered** output stream

I'm programming a little GUI for a file converter in java. The file converter writes its current progress to stdout. Looks like this:

Flow_1.wav: 28% complete, ratio=0,447

I wanted to illustrate this in a progress bar, so I'm reading the process' stdout like this:

ProcessBuilder builder = new ProcessBuilder("...");
builder.redirectErrorStream(true);
Process proc = builder.start();
InputStream stream = proc.getInputStream();
byte[] b = new byte[32];
int length;
while (true) {
    length = stream.read(b);
    if (length < 0) break;
    // processing data
}

Now the problem is that regardless which byte array size I choose, the stream is read in chunks of 4 KB. So my code is being executed until length = stream.read(b); and then blocks for quite a while. Once the process generates 4 KB output data, my programm gets this chunk and works through it in 32 byte slices. And then waits again for the next 4 KB.

I tried to force java to use smaller buffers like this:

BufferedInputStream stream = new BufferedInputStream(proc.getInputStream(), 32);

Or this:

BufferedReader reader = new BufferedReader(new InputStreamReader(proc.getInputStream()), 32);

But neither changed anything.

Then I found this: Process source (around line 87)

It seems that the Process class is implemented in such a way that it pipes the process' stdout to a file. So what proc.getInputStream(); actually does, is returning a stream to a file. And this file seems to be written with a 4 KB buffer.

Does anyone know some kind of workaround for this situation? I just want to get the process' output instantly.

EDIT: As suggested by Ian Roberts, I also tried to pipe the converter's output into the stderr stream, since this stream doesn't seem to be wrapped in a BufferedInputStream. Still 4k chunks.

Another interesting thing is: I actually don't get exactly 4096 bytes, but about 5 more. I'm afraid the FileInputStream itself is buffered natively.

Upvotes: 3

Views: 2147

Answers (1)

Ian Roberts
Ian Roberts

Reputation: 122364

Looking at the code you linked to the process's standard output stream gets wrapped in a BufferedInputStream but its standard error remains unbuffered. So one possibility might be to execute not the converter directly, but a shell script (or Windows equivalent if you're on Windows) that sends the converter's stdout to stderr:

ProcessBuilder builder = new ProcessBuilder("/bin/sh", "-c",
  "exec /path/to/converter args 1>&2");

Don't redirectErrorStream, and then read from proc.getErrorStream() instead of proc.getInputStream().

It may be the case that your converter is already using stderr for its progress reporting in which case you don't need the script bit, just turn off redirectErrorStream(). If the converter program writes to both stdout and stderr then you'll need to spawn a second thread to consume stdout as well (the script approach gets around this by sending everything to stderr).

Upvotes: 2

Related Questions