CODEWITHSUNDEEP
bash
Alex Budovski
Alex Budovski

Reputation: 18446

How to apply shell command to each line of a command output?

Suppose I have some output from a command (such as ls -1):

a
b
c
d
e
...

I want to apply a command (say echo) to each one, in turn. E.g.

echo a
echo b
echo c
echo d
echo e
...

What's the easiest way to do that in bash?

Upvotes: 325

Views: 268427

Answers (10)

anon
anon

Reputation:

This question is a duplicate of Execute a command once per line of piped input? it seems.

I don’t know if it’s appropriate to post an adapted version of the same answer, since it’s the same question, like everyone else did. I’d prefer to merge the questions (any admin reading this?).

But until then, here we go:

What you are asking for is known as a functor. A mapping function.

Since echo isn’t a particularly sensible function to apply things to, since things that go in a pipe are already echoed without that pipe, I’ll use the custom function bla() here.

I also adapted the answer for your ls -1 case.

This should work for everything,

  • including self-defined functions (which xargs can’t do directly),
  • without spawning any additional processes (like xargs does), and
  • without removing spaces (which read otherwise does!).

Note the IFS= and -r, not included in any other answer:

mapp() { while IFS= read -r line; do "$1" "$line"; done; }

Here’s an example usage:

$ bla() { echo "  bla: $1"; }
$ ls -1 | mapp bla 
  bla: a
  bla: b
  bla: c
  …

For alternative versions and other variants, see my answer to the other question.

Upvotes: 0

balupton
balupton

Reputation: 48650

A solution that works with filenames that have spaces in them, is:

ls -1 | xargs -I %s echo %s

The following is equivalent, but has a clearer divide between the precursor and what you actually want to do:

ls -1 | xargs -I %s -- echo %s

Where echo is whatever it is you want to run, and the subsequent %s is the filename.

Thanks to Chris Jester-Young's answer on a duplicate question.

Upvotes: 12

Michael Mrozek
Michael Mrozek

Reputation: 175375

It's probably easiest to use xargs. In your case:

ls -1 | xargs -L1 echo

The -L flag ensures the input is read properly. From the man page of xargs:

-L number
    Call utility for every number non-empty lines read. 
    A line ending with a space continues to the next non-empty line. [...]

Upvotes: 347

Chris
Chris

Reputation: 463

i like to use gawk for running multiple commands on a list, for instance

ls -l | gawk '{system("/path/to/cmd.sh "$1)}'

however the escaping of the escapable characters can get a little hairy.

Upvotes: 3

phil294
phil294

Reputation: 10852

xargs fails with with backslashes, quotes. It needs to be something like

ls -1 |tr \\n \\0 |xargs -0 -iTHIS echo "THIS is a file."

xargs -0 option:

-0, --null
          Input  items are terminated by a null character instead of by whitespace, and the quotes and backslash are
          not special (every character is taken literally).  Disables the end of file string, which is treated  like
          any  other argument.  Useful when input items might contain white space, quote marks, or backslashes.  The
          GNU find -print0 option produces input suitable for this mode.

ls -1 terminates the items with newline characters, so tr translates them into null characters.

This approach is about 50 times slower than iterating manually with for ... (see Michael Aaron Safyans answer) (3.55s vs. 0.066s). But for other input commands like locate, find, reading from a file (tr \\n \\0 <file) or similar, you have to work with xargs like this.

Upvotes: 3

Andrej Pandovich
Andrej Pandovich

Reputation: 326

Better result for me:

ls -1 | xargs -L1 -d "\n" CMD

Upvotes: 1

Łukasz Daniluk
Łukasz Daniluk

Reputation: 470

You actually can use sed to do it, provided it is GNU sed.

... | sed 's/match/command \0/e'

How it works:

  1. Substitute match with command match
  2. On substitution execute command
  3. Replace substituted line with command output.

Upvotes: 11

Trey Hunner
Trey Hunner

Reputation: 11814

You can use a basic prepend operation on each line:

ls -1 | while read line ; do echo $line ; done

Or you can pipe the output to sed for more complex operations:

ls -1 | sed 's/^\(.*\)$/echo \1/'

Upvotes: 282

Marcelo Cantos
Marcelo Cantos

Reputation: 185852

for s in `cmd`; do echo $s; done

If cmd has a large output:

cmd | xargs -L1 echo

Upvotes: 21

Michael Aaron Safyan
Michael Aaron Safyan

Reputation: 95509

You can use a for loop:

for file in * ; do
   echo "$file"
done

Note that if the command in question accepts multiple arguments, then using xargs is almost always more efficient as it only has to spawn the utility in question once instead of multiple times.

Upvotes: 12

Related Questions