CODEWITHSUNDEEP
javalinux
St.Antario
St.Antario

Reputation: 27385

Understanding I/O blocking

I have a binary linux executable which prints some bytes in stdout. I consume these bytes from Java application as this:

String[] cmd;
Process p = Runtime.getRuntime().exec(cmd);
InputStream is = p.getInputStream();
int r = is.read();
while(r != -1){
    System.out.println(r);
    r = is.read(); //1
}

But after some time of working the //1 is blocked for I/O forever (dead-lock). I created thread dump and noticed that

"pool-2-thread-1@627" prio=5 tid=0xd nid=NA runnable
  java.lang.Thread.State: RUNNABLE
      at java.io.FileInputStream.readBytes(FileInputStream.java:-1)
      at java.io.FileInputStream.read(FileInputStream.java:255)
      at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
      - locked <0x2c0> (a java.lang.UNIXProcess$ProcessPipeInputStream)
      at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
      at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
      at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)

It's locked on a native method 

private native int readBytes(byte[] var1, int var2, int var3) throws IOException;

What's a possible reason this method is blocked forever? Maybe this is platform-specific. I'm using Ubuntu 16.04.

Upvotes: 2

Views: 1239

Answers (3)

Frank.Chang
Frank.Chang

Reputation: 993

There are five different type IO model under Linux , they are respectively

  • Blocking IO
  • NonBlocking IO
  • IO Multiplexing
  • Signal-Driven
  • Asynchornours IO

java.io.FileInputStream.readBytes() use first type, Blocking IO. Application waits until kernel returns data. enter image description here

Upvotes: 1

slim
slim

Reputation: 41223

Your InputStream is waiting for the process to either supply another byte, or close the stream (by finishing).

One reason the process might be doing neither of those things, is that it's written to stderr, and is blocking on that write.

Possibly there is a difference in environment between your shell and the environment used by Java, that means the command is resulting in output to stderr (e.g. the program is not in $PATH; you don't have adequate permissions; your CWD isn't what you expect, etc.)

A quick and dirty way to find out if this is the problem, is to redirect stderr to stdout -- either with ProcessBuilder.redirectErrorStream() or by using 2>&1 on the UNIX side of things.

If this is the case, then it is a form of deadlock -- the Java program is waiting for the native program to write to one pipe. The native program is waiting for the Java program to read from another.

The clean way to handle it, which you might choose to do in the longer term, is to use getErrorStream() and handle its output. This isn't exactly trivial, because in a single-threaded program using blocking reads, you can never know which stream is going to have data. You can either do blocking reads in separate threads, or use NIO to handle both inputs in a non-blocking manner.

By the way - note that the Java docs advise ProcessBuilder.start() over Runtime.exec(), since Java 1.5.

Upvotes: 2

John Bollinger
John Bollinger

Reputation: 180266

You don't need to look at the internal methods of the standard library for an answer. The docs of InputStream.read() specify, in part, that

If no byte is available because the end of the stream has been reached, the value -1 is returned. This method blocks until input data is available, the end of the stream is detected, or an exception is thrown.

Perhaps the key point of confusion is the meaning of "the end of the stream is detected." This does not mean that no more bytes are available to read right now -- that wouldn't be consistent with specification that the method blocks until data is available, and in practice it would work out poorly in many situations. Rather, the end of the stream is detected when the system receives some sort of signal that no more bytes will ever be available from it. Generally, that means that the stream has been closed on the source end, and all bytes drained from it at the destination end.

In your particular case, if the external process remains running, does not write anything further to its standard output, yet does not close its standard output, then Java (or a native consumer) will never see end-of-stream on that process's output.

Upvotes: 2

Related Questions