CODEWITHSUNDEEP
linuxbashshell
shaurya
shaurya

Reputation: 65

command not working as expected if run via /bin/sh -c

I have to concatenate a set of files. Directory structure is like this:

root/features/xxx/multiple_files... -> root/xxx/single_file

what i have written (and it works fine):

for dirname in $(ls -d root/features/*|awk -F/ '{print $NF}');do;mkdir root/${dirname};cat root/features/${dirname}/* > root/${dirname}/final.txt;done

But when i run the same thing via sh shell

/bin/sh -c "for dirname in $(ls -d root/features/*|awk -F/ '{print $NF}');do;mkdir root/${dirname};cat root/features/${dirname}/* > root/${dirname}/final.txt;done"

it gives me errors:

/bin/sh: -c: line 1: syntax error near unexpected token `201201000'
/bin/sh: -c: line 1: `201201000'

My process always appends /bin/sh -c before running any commands. Any suggestions what might be going wrong here? Any alternate ways? I have spent a really long time on this ,without making much headway!

EDIT: `ls -d root/features/*|awk -F/ '{print $NF}' returns

201201
201201000
201201001
201201002
201201003
201201004
201201005
201201006
201201007
201202000
201205000
201206000
201207000
201207001
201207002

Upvotes: 1

Views: 11895

Answers (2)

djjolicoeur
djjolicoeur

Reputation: 484

I got this to run in the little test environment I set up on my box. Turns out it didn't like the double quotes. The issue I ran into was the quotes around the awk statement...if you wrap it in double quotes it prints the whole thing.....I used cut to get the desired result, but my guess is you'll have to change the -f arg to 3 instead of 2..I think.

/bin/sh  -c 'for dirname in $(ls -d sh_test/* | awk -F/ '\''{print $NF}'\''); do mkdir sh_test_root/${dirname}; cat sh_test/${dirname}/* > sh_test_root/${dirname}/final.txt;done'

edit: Tested edit proposed by nadu and it works fine. The above reflects that change.

Upvotes: 1

nadu
nadu

Reputation: 71

Always use sh -c 'cmd1 | cmd2' with single quotes.

Always use sh -eu -xv -c 'cmd1 | cmd2' to debug.

Always use bash -c 'cmd1 | cmd2' if your code is Bash-specific (cf. process substitution, ...).

Remove ; after do in for ... ; do; mkdir ....

Escape possible single quotes within single quotes like so: ' --> '\''.

(And sometimes just formatting your code clarifies a lot.)

Applied to your command this should look somewhat like this ...

# test version
/bin/sh -c '
   for dirname in $(ls -d /* | awk -F/ '\''{print $NF}'\''); do
     printf "%s\n" "mkdir root/${dirname}";
     printf "%s\n" "cat root/features/${dirname}/* > root/${dirname}/final.txt"; 
     echo
   done
' | nl

# test version using 'printf' instead of 'ls'
sh -c '
   printf "%s\000" /*/ | while IFS="" read -r -d "" file; do
     dirname="$(basename "$file")"
     printf "%s\n" "mkdir root/${dirname}";
     printf "%s\n" "cat root/features/${dirname}/* > root/${dirname}/final.txt"; 
     echo
   done
' | nl

Upvotes: 7

Related Questions