CODEWITHSUNDEEP
bashiterationpipenamed-pipesfifo
Sparhawk
Sparhawk

Reputation: 1677

How can I use a named pipe as temporary storage in a loop?

I have a loop, where I'm making incremental modifications to a large file. Rather than write to disk each time, I though I'd use named pipes. However, this means that I'll need a unique name for each iteration of the loop, since I can't seem to redirect output back into the same named pipe.

$ mkfifo fifotemp
$ echo qwerty > fifotemp &
$ grep qwe <fifotemp >fifotemp &
$ cat <fifotemp
[hangs]

I could create a new named pipe for each iteration, but this seemed inelegant. Is there a better way?

Upvotes: 0

Views: 481

Answers (1)

Steve Zobell
Steve Zobell

Reputation: 884

Potentially you could use plain pipes and recursive functions. You would need to pass everything into the recursive function to determine when to quit and what processing is needed at each recursion level. This example just adds the recursion level at the front of each line for each level, and quits at level 4:

#!/bin/bash

doEdits() {
   while read -r line
   do
      echo "$1 $line"
   done
}

doRecursion() {
   level=$(($1 + 1))
   #echo "doRecursion $level" 1>&2
   if [ $level -lt 4 ]
   then
      doEdits $level | doRecursion $level
   else
      # Just output all the input
      cat
   fi
}

doRecursion 0 < myInputFile > myOutputFile

I assume the number of recursion levels is fairly limited, otherwise you could run into system limitations on the number of open processes and pipes.

One advantage here is that each pipe should only need a small buffer. This could also be fast if your machine has multiple processors.

Upvotes: 1

Related Questions