CODEWITHSUNDEEP
bash
jjkl
jjkl

Reputation: 403

Reading file from while loop not getting first line

The following attempts to read from a file, gets the 2nd field of a CSV and performs actions if the field value is greater than the first argument specified for the command:

while  read -r line
do
       var="$(cut -f2 -d,)" 
       #echo "$var"
       if [ "$var" -gt "$1" ]; then
         # do stuff
       fi

done < someFile

I'm running into two issues:

  1. The loop is not processing the first line. Am I missing something fundamental about how input redirection works? Or how it works with a while loop?

e.g. If someFile contains:

John,1,green
Bob,2,blue

the commented echo statement would only display 2

  1. The test "$var" -gt "$1" produces the error integer expression expected.

$1 is always a numeric value. I've tried converting $var to an integer using printf but I've been reading there's no need to convert to an integer since -gt does implicit type conversion.

Thanks

Upvotes: 0

Views: 521

Answers (2)

Paul Hodges
Paul Hodges

Reputation: 15246

cut is stealing your data.

$: cat in
x,1,x
x,2,x
x,3,x

$: while read line; do echo "[$line]"; cut -f 2 -d,  ; done < in
[x,1,x]
2
3

The read is reading the first line of my file into $line, which I echo in square brackets so you can see it. Then cut is processing the rest of the file, since both see it as stdin to the whole loop.

What you nead is to explicitly process the data you already read.

$: time while read line; do echo "[$line]"; val=$(cut -f 2 -d,<<<"$line"); echo "<$val>"; done < in
[x,1,x]
<1>
[x,2,x]
<2>
[x,3,x]
<3>

real    0m0.552s
user    0m0.106s
sys     0m0.401s

You can make it a lot faster/more efficient by skipping the cut subprocess and doing it with built-in bash parameter parsing.

$: time while read line; do echo "[$line]"; val="${line#*,}"; val="${val%%,*}"; echo "<$val>"; done < in
[x,1,x]
<1>
[x,2,x]
<2>
[x,3,x]
<3>

real    0m0.001s
user    0m0.000s
sys     0m0.000s

If you don't have an explicit reason to need the entire line with the separators still in it, then all you need is the read.

$: time while IFS=, read v1 v2 v3; do echo "v1=$v1 v2=$v2 v3=$v3"; done < in
v1=x v2=1 v3=x
v1=x v2=2 v3=x
v1=x v2=3 v3=x

real    0m0.001s
user    0m0.000s
sys     0m0.000s

Upvotes: 1

Dominique
Dominique

Reputation: 17493

You read the variable line but you don't use it:

var="$(cut -f2 -d,)"

Shouldn't it be something like:

var="$(cut -f2 -d, $line)"

Or something like that?

Upvotes: 0

Related Questions