CODEWITHSUNDEEP
pythonbashshellsubprocesspipe
thayne
thayne

Reputation: 1038

python subprocess: read from pipe without closing it

I have a process that writes a continuous stream of bytes to stdout. I have another process that reads from the stdout of the first process and then finishes. But I found that when the second process finishes, it closes the pipe (I think) and so process 1 stops running.

I tried this 2 different ways:

in shell 1:

# make fifo
mkfifo /tmp/camera
# stream video from my webcam to the pipe
ffmpeg -y -f v4l2 -r 10 -i /dev/video0 -f avi /tmp/camera

in shell 2:

# attach to pipe and take a snapshot
rrmpeg -y -r 1 -f avi -i /tmp/camera -vframes 1 -s 160x120 -f image2 image1.jpg

I also tried this in shell 2 with the same result:

tail -f /tmp/camera

When the process in shell 2 stops, it kills process 1. Why?

The second thing I tried was with python subprocess:

from subprocess import Popen, PIPE
import shlex, time

stream = Popen(shlex.split("ffmpeg -y -f v4l2 -r 10 -i /dev/video0 -f avi -"), stdout=PIPE)
while True:
    time.sleep(3)
    snap = Popen(shlex.split("rrmpeg -y -r 1 -f avi -i - -vframes 1 -s 160x120 -f image2 image1.jpg"), stdin=stream.stdout)

It works the first time through the loop, but when the snap process goes out of scope, the stream process dies.

Is there any way to read some of the stream from process 1 without closing it?

Upvotes: 2

Views: 1152

Answers (1)

Robert Siemer
Robert Siemer

Reputation: 34667

Yes, there is a way to continue the pipe writer forever. For that, you have to make sure

  1. at least one reader keeps the pipe open (without reading is fine) --or--
  2. the writer ignores SIGPIPE and handles write() errors well (i.e. tries again)

The second option is not so simple (without changing/checking the source of the writer), because the write() on a pipe without reader doesn’t block (once a reader was there).

For the first option, make sure your first reader opens the fifo/pipe, doesn’t read anything from it, then blocks/sleeps on something forever without dying.

Let any successive readers do the work. They can open a fifo (named pipe) or have to be forked from the process having the reading end open (they inherit the filedescriptor).

This way no SIGPIPE is generated, and write() will block the writer, once the pipe buffer is full.

man 7 pipe tells you more.

(Note: some programs might get confused if a write() blocks unexpected long. E.g. some apps which regularly look on the wall-clock.)

Regarding “When the process in shell 2 stops, it kills process 1. Why?”:

Process 1 is probably killed by SIGPIPE (I didn’t check the source), which it gets when it tries write() on a pipe with no reader.

Upvotes: 2

Related Questions