CODEWITHSUNDEEP
bashshellunix
TimCinel
TimCinel

Reputation: 2000

Reuse or repeat pipe input as arguments in command (example given)

I'll explain this problem with an example. Let's say I want to generate a list with two columns. Column 1 contains a file name and column 2 contains the md5 sum of that file.

This can be done using the following script:

for FILE in `ls -p | grep -v "/"`;
do
    printf "%s %s\n" "$FILE" `md5 -q "$FILE"`;
done;

Can this be done on a single line using pipes? I've tried using different combinations of sed, xargs and printf but I think I'm missing something. Here's one of my attempts:

ls -p | grep -v "/" | xargs -I FILE printf "%s %s\n" FILE `md5 -q FILE`

In that attempt, the FILE inside the backticks isn't substituted, which isn't really surprising.

Is this the kind of thing where I should just use a multi-line script? Since there's no logic or control flow, I feel that a one-liner should be possible and that perhaps I'm not using my tools properly or I haven't found the right tool.

Apologies for the title, I have no idea what to call this question.

Upvotes: 4

Views: 2357

Answers (4)

Samveen
Samveen

Reputation: 3540

To take care of concerns regarding file names with weird characters in the name, that require escaping, EXCEPT " character, or file names with md5 sums and spaces as part of their names:

find . -maxdepth 1 -type f  -print0 |xargs -0 -n1 -I{} bash -c 'echo -n "{} " ; cat "{}" |md5sum' | sed 's/ -$//'

Using cat makes the filename for md5sum the standard input (-) which we strip off with sed. However, no complex regexps required to change the order of the columns, as we use echo -n to print the filename as the first column of the output.

Example:

samveen@precise:/tmp/tmp.JpmDNkhcRG$ touch "ab'c"
samveen@precise:/tmp/tmp.JpmDNkhcRG$ touch "ab d41d8cd98f00b204e9800998ecf8427e c"
samveen@precise:/tmp/tmp.JpmDNkhcRG$ touch "ab c"
samveen@precise:/tmp/tmp.JpmDNkhcRG$ touch "abc"
samveen@precise:/tmp/tmp.JpmDNkhcRG$ touch "ab
> c"
samveen@precise:/tmp/tmp.JpmDNkhcRG$ touch "ab\!c"

samveen@precise:/tmp/tmp.JpmDNkhcRG$ find . -maxdepth 1 -type f  -print0 |xargs -0 -n1 -I{} bash -c 'echo -n "{} " ; cat "{}" |md5sum'| sed 's/ -$//'
./ab'c d41d8cd98f00b204e9800998ecf8427e
./ab d41d8cd98f00b204e9800998ecf8427e c d41d8cd98f00b204e9800998ecf8427e
./ab c d41d8cd98f00b204e9800998ecf8427e
./abc d41d8cd98f00b204e9800998ecf8427e
./ab
c d41d8cd98f00b204e9800998ecf8427e
./ab\!c d41d8cd98f00b204e9800998ecf8427e

Upvotes: 1

perreal
perreal

Reputation: 98118

You can use md5sum, because I don't have md5 :)

ls -p | grep -v / | xargs md5sum | awk '{print $2,$1}'

and this may be a safer, more robust way:

find -maxdepth 1 -type f -exec md5sum {} \; | awk '{s=$2; $2=$1; $1=s;}1'

this should work with md5:

find -maxdepth 1 -type f -exec md5sum {} \; | sed 's/[^(]*(\([^)]*\)) =/\1/'

Upvotes: 6

Beta
Beta

Reputation: 99172

How about:

ls *.cc | xargs -I FILE md5 FILE | sed 's/^MD5 (\(.*\)) =/\1/'

Or if you're worried about spaces in your filenames (which I don't think you should allow yourself in any case),

find . -name "*.cc" -maxdepth 1 | xargs -I FILE md5 FILE | sed 's/^MD5 (\(.*\)) =/\1/'

Upvotes: 3

Barmar
Barmar

Reputation: 782509

ls -p | grep -v "/" | xargs -I FILE sh -c 'printf "%s %s\n" $0 `md5 -q $0`' FILE

Your code wasn't working because the backticks are processed by the original shell before passing the arguments to xargs. You need to protect them from processing by enclosing them in quotes, and then passing that argument to sh.

However, this seems like a bad way to do it. It doesn't work for files with spaces in the names, and grep is a silly way to skip directories.

for FILE in *
do
  if  [ -f "$FILE" ]
  then printf "%s %s\n" "$FILE" `md5 -q "$FILE"`
done

Upvotes: 1

Related Questions