CODEWITHSUNDEEP
clinuxstdio
tay10r
tay10r

Reputation: 4337

how to communicate to a program with another external program

I'm trying to write to stdin and read from stdout ( and stderr ) from an external program, without changing the code.

I've tried using named pipes, but stdout doesn't show until the program is terminated and stdin only works on the first input( then cin is null ).

i've tried using /proc/[pid]/fd but that only writes and reads from the terminal and not the program.

i've tried writing a character device file for this and it worked, but only one program at a time ( this needs to work for multiple programs at a time ).

at this point, to my knowledge, I could write the driver that worked to multiplex the io across multiple programs but I don't think that's the "right" solution.

the main purpose of this is to view a feed of a program through a web interface. I'm sure there has to be someway to do this. is there anything I haven't tried that's been done before?

Upvotes: 2

Views: 568

Answers (1)

Adam Rosenfield
Adam Rosenfield

Reputation: 400156

The typical way of doing this is:

  1. Create anonymous pipes (not named pipes) with the pipe(2) system call for the new process's standard streams
  2. Call fork(2) to spawn the child process
  3. close(2) the appropriate ends of the pipes in both the parent and the child (e.g. for the stdin pipe, close the read end in the parent and close the write end in the child; vice-versa for the stdout and stderr pipes)
  4. Use dup2(2) in the child to copy the pipe file descriptors onto file descriptors 0, 1, and 2, and then close(2) the remaining old descriptors
  5. exec(3) the external application in the child process
  6. In the parent process, simultaneously write to the child's stdin pipe and read from the child's stdout and stderr pipes. However, depending on how the child behaves, this can easily lead to deadlock if you're not careful. One way to avoid deadlock is to spawn separate threads to handle each of the 3 streams; another way is to use the select(2) system call to wait until one of the streams can be read from/written to without blocking, and then process that stream.

Even if you do this all correctly, you may still not see your program's output right away. This is typically due to buffering stdout. Normally, when stdout is going to a terminal, it's line-buffered—it gets flushed after every newline gets written. But when stdout is a pipe (or anything else that's not a terminal, like a file or a socket), it's fully buffered, and it only gets written to when the program has outputted a full buffer's worth of data (e.g. 4 KB).

Many programs have command line options to change their buffering behavior. For example, grep(1) has the --line-buffered flag to force it to line-buffer its output even when stdout isn't a terminal. If your external program has such an option, you should probably use it. If not, it's still possible to change the buffering behavior, but you have to use some sneaky tricks—see this question and this question for how to do that.

Upvotes: 5

Related Questions