string formatting in bash

Question

I have the following code that runs in a shell script

foo=`seq 1 1 100`
for i in $foo; do
  echo "input$i\_now"
done

Here's my question: Under some conditions, the output prints input1_now whereas othertimes it prints input1\_now. I'm sure something is different, but I can't figure out what makes it print one way or the other. If my code is

for i in $foo; do
   echo "input$i_now"
done

I will always get input with the rest of the line being omitted.

I know I can use input${i}_now instead and have it print correctly every time, but I'm mostly interested in understanding why the output is different under seemingly the same conditions.

UPDATE:

In the following example, the first part correctly formats the variables and text such that the \_ is replaced as _. However, the last part required me to place variables in curly brackets in order to have them formatted correctly.

echo "Enter Simulation #: "
read sim

g.mapset results

for i in `seq 1 1 100`; do

file=sim$sim\_run$i\_sum
g.copy $file\@expSim$sim\_$i,$file

file=sim$sim\_run$i\_average
g.copy $file\@expSim$sim\_$i,$file

for year in `seq 2004 1 2006`; do   

    file=sim$sim\_$year\_run$i\_sum
    g.copy $file\@expSim$sim\_$i,$file

    file=sim$sim\_$year\_run$i\_average
    g.copy $file\@expSim$sim\_$i,$file

done

years="2004 2005 2006"
times=`seq -w 1 16 365`
runs=`seq 1 1 100`

for year in $years; do
for ptime in $times; do
    for i in $runs; do
        if [ $i -eq 1 ]; then
            g.copy  vect=sim${sim}_pts_${year}_${ptime}_run${i}@expSim${sim}_${i},sim${sim}_pts_${year}_${ptime}
        fi
        if [ $i -gt  1 ]; then  
        v.patch input=sim${sim}_pts_${year}_${ptime}_run${i}@expSim${sim}_${i} output=sim${sim}_pts_${year}_${ptime} -e -a --o
        fi
    done
done
done

Answer 1

Underscore is a valid character in bash variable names so you need to protect the variable name from surrounding characters.

foo=`seq 1 1 100`
for i in $foo; do
  echo "input${i}_now"
done

The difference:

input$i_now    # "input" + $i_now
input${i}_now  # "input" + $i + "_now"
input"$i"_now  # input + "$i" + now
input$i\_now   # input + $1 + _now - \_ "escapes" the underscore

https://man7.org/linux/man-pages/man1/bash.1.html

name A word consisting only of alphanumeric characters and underscores, and beginning with an alphabetic character or an underscore. Also referred to as an identifier.

You have to protect the parameter name, either by PREFIX${braces}SUFFIX, with PREFIX"$quotes"SUFFIX or by escaping - \ - characters which are valid in parameter names. All "work" but I prefer braces which is also what the documentation refers to:

The parameter name or symbol to be expanded may be enclosed in braces, which are optional but serve to protect the variable to be expanded from characters immediately following it which could be interpreted as part of the name.

Answer 2

Is _ supposed to be a placeholder that is sometimes a different character?

In bash, "input$i\_now" with an actual _ will always produce input1\_now. Inside double-quotes, bash only removes a \ when it is followed by a $, a `, a ", a \, or a newline. See “Double Quotes” in the Bash Reference Manual. This is the POSIX standard behavior; see “Double-Quotes” in Shell Command Language.

UPDATE

If you write "input$i_now", bash will just print input. It will not print input1 or input1_now. It does this because _ is a valid parameter name character, so bash thinks you are asking for the value of the i_now parameter. Unless you have set i_now to a non-empty string, bash will expand $i_now to the empty string, thus turning "input$i_now" into input.

UPDATE 2

Now that you have posted real code, we can see what's going on.

First of all, in the real code you posted, you never used double-quotes around a parameter expansion. This makes a difference.

Outside of double-quotes, a \ is always removed. See “Quote Removal” in the Bash Reference Manual. Hence input$i\_now (with no surrounding double-quotes) expands to input1_now.

However, as I explained in my first update, _ is a parameter name character. See “Name” in Shell Command Language. So when bash sees input$i_now, it takes i_now as the parameter name.

Whether or not you're using double-quotes, you must separate the parameter name from the following character, if bash would otherwise treat the following character as part of the parameter name. You can do this by putting \ after the parameter name, or you can do it by putting the parameter name in {...}.

It is safer to always use {...}, because (as you have discovered?) \ is handled differently depending on whether it's inside double-quotes. If you go back and add double-quotes later, and you have used \, you will need to change the \ to {...} anyway.

UPDATE 3

Here is a demonstration of the effects of \, {...}, and double-quoting. First, we set up some variables:

$ year=2004 ptime=1 i=1 sim=123

Here's what happens with no quoting whatsoever:

$ echo vect=sim$sim_pts_$year_$ptime_run$i@expSim$sim_$i,sim$sim_pts_$year_$ptime
vect=sim1@expSim1,sim1

Here's what happens if we just use {...} without double-quotes:

$ echo vect=sim${sim}_pts_${year}_${ptime}_run${i}@expSim${sim}_${i},sim${sim}_pts_${year}_${ptime}
vect=sim123_pts_2004_1_run1@expSim123_1,sim123_pts_2004_1

If we add double-quotes, they have no effect:

$ echo "vect=sim${sim}_pts_${year}_${ptime}_run${i}@expSim${sim}_${i},sim${sim}_pts_${year}_${ptime}"
vect=sim123_pts_2004_1_run1@expSim123_1,sim123_pts_2004_1

Here's what happens if we just use \:

$ echo vect=sim$sim\_pts\_$year\_$ptime\_run$i@expSim$sim\_$i,sim$sim\_pts\_$year\_$ptime
vect=sim123_pts_2004_1_run1@expSim123_1,sim123_pts_2004_1

Notice that each \ was removed. The shell removes a \ if it's not quoted.

If we add double-quotes, they prevent the shell from removing each \:

$ echo "vect=sim$sim\_pts\_$year\_$ptime\_run$i@expSim$sim\_$i,sim$sim\_pts\_$year\_$ptime"
vect=sim123\_pts\_2004\_1\_run1@expSim123\_1,sim123\_pts\_2004\_1