CODEWITHSUNDEEP
linuxbash
Sam
Sam

Reputation: 130

Why does a part of this variable get replaced when combining it with a string?

I have the following Bash script which loops through the lines of a file:

INFO_FILE=playlist-info-test.txt
line_count=$(wc -l $INFO_FILE | awk '{print $1}')

for ((i=1; i<=$line_count; i++))
do
    current_line=$(sed "${i}q;d" $INFO_FILE)

    CURRENT_PLAYLIST_ORIG="$current_line"
    input_file="$CURRENT_PLAYLIST_ORIG.mp3"
    echo $input_file
done

This is a sample of the playlist-info-test.txt file:

Playlist 1
Playlist2
Playlist 3

The output of the script should be as follows:

Playlist 1.mp3
Playlist2.mp3
Playlist 3.mp3

However, I am getting the following output:

.mp3list 1
.mp3list2
.mp3list 3

I have spent a few hours on this and can't understand why the ".mp3" part is being moved to the front of the string. I initially thought it was because of the space in the lines of the input file, but removing the space doesn't make a difference. I also tried using a while loop with read line and the input file redirected into it, but that does not make any difference either.

Upvotes: 0

Views: 60

Answers (2)

Gordon Davisson
Gordon Davisson

Reputation: 125788

Did the file come from Windows? DOS and Windows end their lines with carriage return (hex 0d, sometimes represented as \r) followed by linefeed (hex 0a, sometimes represented as \n). Unix just uses linefeed, and so tends to treat the carriage return as part of the content of the line. In your case, it winds up at the end of the current_line variable, so input_file winds up something like "Playlist 1\r.mp3". When you print this to the terminal, the carriage return makes it go back to the beginning of the line (that's what carriage return means), so it prints as:

Playlist 1
.mp3

...with the ".mp3" printed over the "Play" part, rather than on the next line like I have it above.

Solution: either fix the file (there's a fairly standard dos2unix program that does precisely this), or change your script to strip carriage returns as it reads the file. Actually, I'd recommend a rewrite anyway, since your current use of sed to pick out lines is rather weird and inefficient. In a shell script, the standard way to read through a file line-by-line is to use a loop like while read -r current_line; do [commands here]; done <"$INFO_FILE". There's a possible problem that if any commands inside the loop read from standard input, they'll wind up inhaling part of that file; you can fix that by passing the file over unit 3 rather than standard input. With that fix and a trick to trim carriage returns, here's what it looks like:

INFO_FILE=playlist-info-test.txt

while IFS=$' \t\n\r' read -r current_line <&3; do
    CURRENT_PLAYLIST_ORIG="$current_line"
    input_file="$CURRENT_PLAYLIST_ORIG.mp3"
    echo "$input_file"
done 3<"$INFO_FILE"

(The carriage return trim is done by read -- it always auto-trims leading and trailing whitespace, and setting IFS to $' \t\n\r' tells it to treat spaces, tabs, linefeeds, and carriage returns as whitespace. And since that assignment is a prefix to the read command, it applies only to that one command and you don't have to set IFS back to normal afterward.)

A couple of other recommendations while I'm here: double-quote all variable references (as I did with echo "$input_file" above), and avoid all-caps variable names (there are a bunch with special meanings, and if you accidentally use one of those it can have weird effects). Oh, and try passing your scripts to shellcheck.net -- it's good at spotting common mistakes.

Upvotes: 0

l0b0
l0b0

Reputation: 58788

I copied the playlist-info-test.txt contents and the script, and get the output you expected. Most likely there are non-printable characters in your playlist-info-test.txt or script which are messing up the processing. Check the binary contents of both files using for example xxd -g 1 and look for non-newline (0a) non-printing characters.

Upvotes: 2

Related Questions