CODEWITHSUNDEEP
linuxbash
Felipe Sodre Silva
Felipe Sodre Silva

Reputation: 221

Best way to pipe a program that reads only one line of input

Suppose I have a command cmd1 that reads one line of input from standard input and produces one line of output. I also have another command cmd2 which produces multiple lines of output. How do I pipe these two commands in linux so that cmd1 is executed for each line produced by cmd2? If I simply do:

# cmd2 | cmd1

cmd1 will take only the first line of output from cmd2, produce one line of output and then close. I know I can use an interpreter like perl to do the job, but I wonder if there's a clean way to do it using bash script only.

Thanks

Upvotes: 3

Views: 3027

Answers (4)

CaptainKirk
CaptainKirk

Reputation: 1

cmd2 | awk '{ system( "echo " $1 " | cmd1" ) }'

A usable example, I validate the ownership/permissions of the .htaccess files across all sites.

find /var/www/html/ -name .htaccess | awk '{ system( "ls -lA " $1 ) }'

Upvotes: 0

Ole Tange
Ole Tange

Reputation: 33685

I have a feeling that multiple cmd1 can be run in parallel.

If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:

cmd2 | parallel --pipe -N1 cmd1

You can install GNU Parallel simply by:

wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
cp parallel sem

Watch the intro videos for GNU Parallel to learn more: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Upvotes: 1

Mat
Mat

Reputation: 206729

You could use a while loop like this:

#! /bin/bash
IFS=$'\n'
while read -r line ; do
    echo "$line" | cmd1
done < <(cmd2)

Should preserve whitespace in lines. (The -r in read is to go into "raw" mode to prevent backslash interpretation.)

Upvotes: 7

chepner
chepner

Reputation: 531275

cmd2 | while read line; do echo $line | cmd1; done

Upvotes: 1

Related Questions