UNIX shell script do loop execute commands

Question

In general I don't understand how to make most commands in a UNIX shell script do loop work the same as they work directly from the command line (using bash).

As a simple test, a script called looping.sh to execute an SQL script (what's in filelist.txt doesn't matter in this case):

for i in $(cat filelist.txt) 
do  $(sqlplus DB_USER/password@abc @test.sql) 
done

results in

looping.sh: line 2: SQL*Plus:: command not found

for each line in filelist.txt. Other variations on the 2nd line don't work, like putting it in quotes etc.

Or, if filelist.txt has names of other sh scripts, let's say a single line in this case called_file1.sh and I want to execute it

for i in $(cat filelist.txt) 
do  exec $i
done

results in

: not found line 2: exec: called_file1.sh

The files are all in the same folder. I tried variations for the second line like /bin/sh $i, putting it in quotes and so on. What's the magic way to execute a command in the do loop?

Answer 1

$(...) takes the contents and runs it as a command and then returns the output from the command.

So when you write:

for i in $(cat filelist.txt) 
do  $(sqlplus DB_USER/password@abc @test.sql) 
done

what the shell does when it hits the body of the loop is run sqlplus DB_USER/password@abc @test.sql and then it takes the output from that command (whatever it may be) and replaces the $(...) bit with it. So you end up with (not exactly since it happens again every loop but for sake of illustration) a loop that looks like this:

for i in $(cat filelist.txt) 
do  <output of 'sqlplus DB_USER/password@abc @test.sql' command>
done

and if that output isn't a valid shell command you are going to get an error.

The solution there is to not do that. You don't want the wrapping $() there at all.

for i in $(cat filelist.txt) 
do  sqlplus DB_USER/password@abc @test.sql
done

In your second example:

for i in $(cat filelist.txt) 
do  exec $i
done

you are telling the shell that the filename in $i is something that it should try to execute like a binary or executable shell script.

In your case two things are happening here. The filename in $i can't be found and (and this is harder to notice) the filename in $i contains a carriage-return at the end (probably a DOS line-ending file). (That's why the error message is a bit more confused then normal.) (I actually wonder about that since I wouldn't have expected that from an unquoted $i but from a quoted "$i" but I might just be wrong about that.)

So, for this case, you need to both strip the carriage-returns from the file (see point 1 of the "Before asking about problematic code" section of the bash tag info wiki for more about this) and then you need to make sure that filename is an executable script and you have the correct path to it.

Oh, also, exec never returns so that loop will only execute one file ever.

If you want multiple executions then drop exec.

That all being said you Don't Read Lines With For. See Bash FAQ 001 for how to correctly (and safely) read lines from a file.