Redirecting the output of a process into the input of another process using ProcessBuilder in Java

Question

I have two processes defined by processBuilders:

ProcessBuilder pb1 = new ProcessBuilder (...)
ProcessBuilder pb2 = new ProcessBuilder (...)

I want the output of pb1 to be the input to pb2. I found in the documentation that I can make the input of pb2 to be read from another process by using pipe:

pb2.redirectInput(Redirect.PIPE);

However, how can I specify that I want this pipe to read from the output of pb1?

Answer 1

in Android, you can create piped ProcessBuilder with fifo (named pipe) file

    static File createTempFifo() throws IOException, ErrnoException {
        File fifoFile = File.createTempFile("PipedProcessBuilders", ".fifo");
        fifoFile.delete();
        Os.mkfifo(fifoFile.getPath(), 0666);
        return fifoFile;
    }

    static void safeClose(Closeable closeable) {
        try {
            closeable.close();
        } catch (IOException e) {
            // ignore
        }
    }

    @RequiresApi(api = Build.VERSION_CODES.O)
    static Process[] startPipelineProcessBuilder(ProcessBuilder... builders) throws IOException, ErrnoException {
        final Process[] processes = new Process[builders.length];
        final File[] fifoFiles = new File[builders.length - 1];
        final RandomAccessFile[] randomAccessFiles = new RandomAccessFile[builders.length - 1];
        try {
            for (int i = 0; i < builders.length; i++) {
                if (i < builders.length - 1) {
                    fifoFiles[i] = createTempFifo();
                    // because linux require the first opener must open the fifo with Read&Write permission
                    // otherwise the open method will block!
                    randomAccessFiles[i] = new RandomAccessFile(fifoFiles[i], "rw");
                    builders[i].redirectOutput(fifoFiles[i]);
                }
                if (i > 0) {
                    builders[i].redirectInput(fifoFiles[i - 1]);
                }
                processes[i] = builders[i].start();
            }
        } finally {
            Arrays.stream(fifoFiles).filter(Objects::nonNull).forEach(File::delete);
            Arrays.stream(randomAccessFiles).filter(Objects::nonNull).forEach(Client::safeClose);
        }
        return processes;
    }

or use the anonymous pipe

    static Process[] startPipelineProcessBuilder(ProcessBuilder... builders) throws IOException {
        final Process[] processes = new Process[builders.length];
        ParcelFileDescriptor[] newPipes = null, oldPipes = null;
        for (int i = 0; i < processes.length; i++) {
            if (i < processes.length - 1) {
                newPipes = ParcelFileDescriptor.createPipe();
                builders[i].redirectOutput(new File("/proc/self/fd/" + newPipes[1].getFd()));
            }
            if (i > 0) {
                builders[i].redirectInput(new File("/proc/self/fd/" + oldPipes[0].getFd()));
            }
            processes[i] = builders[i].start();
            if (i > 0) {
                oldPipes[0].close();
                oldPipes[1].close();
            }
            oldPipes = newPipes;
        }
        return processes;
    }

test code

        Process[] processes1 = startPipelineProcessBuilder(new ProcessBuilder("busybox", "seq", "1"));
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(processes1[processes1.length - 1].getInputStream()))) {
            assertEquals("1", reader.readLine());
        }
        Process[] processes2 = startPipelineProcessBuilder(new ProcessBuilder("busybox", "seq", "10000"), new ProcessBuilder("wc", "-l"));
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(processes2[processes2.length - 1].getInputStream()))) {
            assertEquals("10000", reader.readLine());
        }
        assertThrows(IOException.class, ()->{
            startPipelineProcessBuilder(new ProcessBuilder("busybox","seq","10000"), new ProcessBuilder("not_exist_file"));
        });

Answer 2

As of JDK 9, you can use startPipeline like so:

ProcessBuilder.startPipeline(
    Arrays.asList(
        new ProcessBuilder(...),
        new ProcessBuilder(...),
        ...
    )
)

Answer 3

static ProcessBuilder.Redirect INHERIT Indicates that subprocess I/O source or destination will be the same as those of the current process.

static ProcessBuilder.Redirect PIPE Indicates that subprocess I/O will be connected to the current Java process over a pipe.

So I don't think one of these will help you redirecting the output of one process to the input of another process. You have to implement it yourself.

Implementation:

public class RedirectStreams {
public RedirectStreams(Process process1, Process process2) {
    final Process tmpProcess1 = process1;
    final Process tmpProcess2 = process2;
    new Thread(new Runnable() {
        @Override
        public void run() {
            BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(tmpProcess1.getInputStream()));
            BufferedWriter bufferedWriter = new BufferedWriter(new OutputStreamWriter(tmpProcess2.getOutputStream()));
            String lineToPipe;

            try {

                while ((lineToPipe = bufferedReader.readLine()) != null){
                    System.out.println("Output process1 / Input process2:" + lineToPipe);
                    bufferedWriter.write(lineToPipe + '\n');
                    bufferedWriter.flush();
                }

            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }).start();
}

}

This one can surely be designed nicer and I haven't tested, if it runs's safe, but it does the job.

Usage:

RedirectStreams redirectStreams = new RedirectStreams(process1,process2);

Test:

public class ProcessPipeTest {
@Test public void testPipe(){
    try {

        Process process1 = new ProcessBuilder("/bin/bash").start();
        Process process2 = new ProcessBuilder("/bin/bash").start();

        RedirectStreams redirectStreams = new RedirectStreams(process1,process2);

        
        BufferedWriter bufferedWriter = new BufferedWriter(new OutputStreamWriter(process1.getOutputStream()));
        String command = "echo echo echo";
        System.out.println("Input process1: " + command);
        bufferedWriter.write(command + '\n');
        bufferedWriter.close();

        
        BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(process2.getInputStream()));
        String actualOutput = bufferedReader.readLine();
        System.out.println("Output process2: " + actualOutput);
        assertEquals("echo",actualOutput);

    } catch (IOException e) {
        e.printStackTrace();
    }
}
}

Output:

Input process1: echo echo echo

Output process1 / Input process2:echo echo

Output process2: echo