CODEWITHSUNDEEP
linuxshellawkbash
jwillis0720
jwillis0720

Reputation: 4477

AWK or bash profile is adding unwanted color codes to stdout

Hi I'm using AWK to produce a bunch of submissions to our cluster.

$ls myfiles* | awk '{for(i=1;i<101;i++){print("qsub -d `pwd` -v FILE="$1",NUM="i" -N "$1" run.qsub")}}'

produces something like - 

qsub -d `pwd` -v FILE=myfiles100.txt,NUM=100 -N myfiles100.txt run.qsub

EXCEPT the variable $1 (myfiles100.txt) that was substituted in awk statement, is highlighted in green syntax. I don't know if this is due to my bash profile or something with AWK, but I've never seen it before. The problem comes when I redirect this to stdout.

$ls myfiles* | awk '{for(i=1;i<101;i++){print("qsub -d `pwd` -v FILE="$1",NUM="i" -N "$1" run.qsub")}}' > somefile.txt

And when I open somefile.txt

qsub -d `pwd` -v FILE=^[[0m^[[32mmyfiles100.txt^[[0m,NUM=100 -N ^[[0m^[[32myfiles100.txt^[[0m run.qsub

The color codes are inserted as well, but this causes confusion when I execute these jobs to the scheduler. I can remove the color codes with a nice sed command.

sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g"

But there has to be some setting I'm missing to make things easier.

Upvotes: 1

Views: 256

Answers (2)

gniourf_gniourf
gniourf_gniourf

Reputation: 46903

You're not supposed to parse the output of ls.

In your specific case, it's much better to just do this:

printf '%s\n' myfiles* | awk '{for(i=1;i<101;i++){print("qsub -d `pwd` -v FILE="$1",NUM="i" -N "$1" run.qsub")}}'

Here's a pure Bash possibility to achieve what you want:

for file in myfiles*; do
    for i in {1..100}; do
        printf 'qsub -d "$PWD" -v FILE=%q,NUM=%d -N %q run.qsub\n' "$file" "$i" "$file"
    done
done > somefile.txt

where I used %q to print out the filename, just in case the filename contains spaces or other funny symbols: these will be properly quoted (of course, this assumes that your commands will be executed by Bash on the cluster). I also used "$PWD" instead of your `pwd`, so that you'll save a subshell each time the command is executed.

Upvotes: 2

Dump Cake
Dump Cake

Reputation: 300

It sounds like the ls you're using is an alias for ls --color . You can check this with which ls. One way around this would be to specify the full path when using ls. In most cases it's /usr/bin/ls.

Upvotes: 1

Related Questions