CODEWITHSUNDEEP
linuxbash
user385948
user385948

Reputation: 909

Read line by line in Bash script

I want to do the following: Read a file line by line and use the line as a parameter.

FILE="cat test"
echo "$FILE" | \
while read CMD; do
echo $CMD
done

But when I do echo $CMD, it just prints cat test.

Upvotes: 74

Views: 210698

Answers (7)

John Kugelman
John Kugelman

Reputation: 361605

The best way to do this is to redirect the file into the loop:

# Basic idea. Keep reading for improvements.
FILE=test

while read CMD; do
    echo "$CMD"
done < "$FILE"

A redirection with < "$FILE" has a few advantages over cat "$FILE" | while .... It avoids a useless use of cat, saving an unnecessary child process. It also avoids a common pitfall where the loop runs in a subshell. In Bash, commands in a | pipeline run in subshells, which means variable assignments are lost after the loop ends. Redirection with < doesn't have that problem, so you could use $CMD after the loop or modify other variables inside the loop. It also, again, avoids unnecessary child processes.

There are some additional improvements that could be made:

  • Add IFS= so that read won't trim leading and trailing whitespace from each line.
  • Add -r to read to prevent backslashes from being interpreted as escape sequences.
  • Lower-case CMD and FILE. The Bash convention is that only environmental and internal shell variables are uppercase.
  • Use printf in place of echo which is safer if $cmd is a string like -n, which echo would interpret as a flag.
file=test

while IFS= read -r cmd; do
    printf '%s\n' "$cmd"
done < "$file"

Upvotes: 155

Mark Yuan
Mark Yuan

Reputation: 21

while read CMD; do
    echo $CMD
done  << EOF
data line 1
data line 2
..
EOF

Upvotes: 1

KomodoDave
KomodoDave

Reputation: 7319

xargs is the most flexible solution for splitting output into command arguments.

It is also very human readable and easy to use due to its simple parameterisation.

Format is xargs -n $NUMLINES mycommand.

For example, to echo each individual line in a file /tmp/tmp.txt you'd do:

cat /tmp/tmp.txt | xargs -n 1 echo

Or to diff each successive pair of files listed as lines in a file of the above name you'd do:

cat /tmp/tmp.txt | xargs -n 2 diff

The -n 2 instructs xargs to consume and pass as separate arguments two lines of what you've piped into it at a time.

You can tailor xargs to split on delimiters besides carriage return/newline.

Use man xargs and google to find out more about the power of this versatile utility.

Upvotes: 16

Umut Utkan
Umut Utkan

Reputation: 648

The correct version of your script is as follows;

FILE="cat test"
$FILE | \
while read CMD; do
echo $CMD
done

However this kind of indirection --putting your command in a variable named FILE-- is unnecessary. Use one of the solutions already provided. I just wanted to point out your mistake.

Upvotes: -1

Santiago Alessandri
Santiago Alessandri

Reputation: 6855

If you want to use each of the lines of the file as command-line params for your application you can use the xargs command.

xargs -a <params_file> <command>

A params file with:

a
b
c
d

and the file tr.py:

import sys
print sys.argv

The execution of 

xargs -a params ./tr.py

gives the result:

['./tr.py', 'a', 'b', 'c', 'd']

Upvotes: 3

Joel
Joel

Reputation: 30156

Do you mean to do:

cat test | \
while read CMD; do
echo $CMD
done

Upvotes: 4

Oliver Charlesworth
Oliver Charlesworth

Reputation: 272497

What you have is piping the text "cat test" into the loop.

You just want:

cat test | \
while read CMD; do
    echo $CMD
done

Upvotes: 103

Related Questions