CODEWITHSUNDEEP
javamultithreadingprocess
Exploring
Exploring

Reputation: 3499

ProcessBuilder is holding a lock to the spawned process

I am invoking an external JAVA process from my program. I am consuming the output generated by that spawned process like this:

DataInputStream dis = new DataInputStream(new BufferedInputStream(myProcess.getInputStream()));

From a Thread I am doing the following:

            while (dis.available() != 0) 
            {                   

                firstMesg = dis.readLine();


                if(firstMesg != null) 
                {
                    // processing with the message
                    //System.out.println(firstMesg);    
                }                                                                       
            } 


            try 
            {
                Thread.currentThread().sleep(SLEEP_TIME);
            }
            catch(Throwable e) 
            {
            }

I was giving SLEEP_TIME around 1 minute and everything was working just fine. Suddenly for a particular setup I have found Sys out (System.out.println) is taking awefully long time from the spawned process.

Can anyone point to me what happened? These two processes must be independent. However the invoker is holding reading from the invoked process. But the buffer should be large where the invoked process is writing. So there is no way it should get blocked.

I can see this in the ProcessBuilder Java doc:

The parent process uses these streams (#getInputStream(), #getErrorStream()) to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.

Upvotes: 0

Views: 759

Answers (3)

brettw
brettw

Reputation: 11124

Take a look at the NuProcess library, it provides non-blocking (asynchronous) I/O for spawned processes. Disclaimer: I am the author of NuProcess.

Upvotes: 1

rolfl
rolfl

Reputation: 17697

This is behaviour I would expect in your case. If a process produces stdout (or stderr for that matter) output it can't continue doing that forever without something actually doing something with that output. As your process produces output it will get buffered by the operating system until that buffer gets full. Your process reads from the buffer making more space again. If your program does not read then at some point the buffer will fill up, and the operating system will suspend the process until space comes available.

Create a thread in your Java process to read the STDOUT, and buffer it up in the Java side if you need to. On the other hand, you should probably process it as it arrives simply because it sounds like there is a lot of it.

EDIT: Based on comments... and other things....

DataInputStream.readLine() is deprecated, don't use it.

Instead of having a while loop that does:

while (dis.available() != 0) {...}

do

while ((line = br.readLine()) != null) { ....}

where br is a new BufferedReader(new InputStreamReader(myProcess.getInputStream()))

Upvotes: 0

Brian Agnew
Brian Agnew

Reputation: 272437

You need to continually read, rather than read what's immediately available and then sleep. Note (also) that you should do the same for both stdout and stderr.

See this question for info on using a StreamGobbler to achieve this.

Upvotes: 0

Related Questions