CODEWITHSUNDEEP
clinuxpipeflush
ern0
ern0

Reputation: 3172

Is there a way to set up a Linux pipe to non-buffering or line-buffering?

My program is controlling an external application on Linux, passing in input commands via a pipe to the external applications stdin, and reading output result via a pipe from the external applications stdout.

The problem is that writes to pipes are buffered by block, and not by line, and therefore delays occur before my app receives data output by the external application. The external application cannot be altered to add explicit fflush() calls.

When I set the external application to /bin/cat -n (it echoes back the input, with line numbers added), it works correctly, it seems, cat flushes after each line. The only way to force the external application to flush, is sending exit command to it; as it receives the command, it flushes, and all the answers appears on the stdout, just before exiting.

I'm pretty sure, that Unix pipes are appropiate solution for that kind of interprocess communication (pseudo server-client), but maybe I'm wrong.

(I've just copied some text from a similar question: Force another program's standard output to be unbuffered using Python)

Upvotes: 4

Views: 5190

Answers (4)

Employed Russian
Employed Russian

Reputation: 213799

Using a PTY may be an overkill for the problem at hand (although it will work).

If the "target application" (the Delphi command-line utility) is dynamically linked, a possibly much simpler solution is to interpose (via LD_PRELOAD) a small library into the application. That library simply needs to implement isatty, and answer true (return 1) regardless of whether the output is going to a pipe or a terminal. You may wish to do that for all file descriptors, or just for STDOUT_FILENO.

Most UNIX implementations will call isatty to decide whether to do full buffering or line buffering for a given file descriptor.

Hmm, glibc doesn't. It calls __fxstat, and then only calls isatty if the status indicates that fd is going to a character device. So you'll need to interpose both __fxstat and isatty. More on library interposition here.

Upvotes: 2

A.H.
A.H.

Reputation: 66263

By default standard input and standard output are fully buffered unless they are connected to an interactive device in which cases they are line buffered [1]. Pipes are non-interactive devices. PTYs are interactive devices. "Fully buffered" means "use a chunk of memory of a certain size".

I'm sure you want line buffering. Therefore using a master/slave PTY instead of pipes should bring the controlled application into the right buffering mode automatically.

[1] see "stdin(3)" and "setbuf(3)" for details.

Upvotes: 1

Basile Starynkevitch
Basile Starynkevitch

Reputation: 1

Why calling fflush suitably (on the write side) don't work for you?

You can use poll (or other syscalls like ppoll, pselect, select) to check availability of input on the read side.

If the external application is using <stdio.h> without calling fflush appropriately (perhaps by setbuf making it happen on newlines ....), data would remain inside its FILE* buffer without even being sent (with a write syscall) to the pipe!

An application can detect if its output is a terminal with e.g. isatty. But it should ensure that flushing happens...

As Michael Dillon suggested, using pty-s is probably the best. But it is hard (I forgot the gory details).

Upvotes: 0

Michael Dillon
Michael Dillon

Reputation: 32392

Don't use a pipe. Use a pty instead. Pty's (pseudo-ttys) have the benefit of being line buffered if you want it, which provides you with simple framing for your data stream.

Upvotes: 5

Related Questions