CODEWITHSUNDEEP
bashshellcsv
nkm
nkm

Reputation: 53

Read From Multiple CSV Files in a loop in shell

I have a requirement of reading some CSV files one by one, CSV files will be named Test1.csv,Test2.csv, etc. The code:

     #!/bin/bash
     IFS=","
     FILES=/CSVFiles/*
     CSVNAME=Test
     n=1
     for f in $FILES 
    while read Column1 Column2 
    do
            echo $Column1
            echo $Column2
    done < "$CSVNAME"$n.csv
    n=$((n+1))

Returns these errors:

./ReadCSV.sh: line 23: syntax error near unexpected token `while'

./ReadCSV.sh: line 23: `while read Column1 Column2

Upvotes: 0

Views: 5854

Answers (3)

sjsam
sjsam

Reputation: 21955

You're missing a 

do

after

for f in $FILES

But if I understood your intentions correctly, you could improve the example using the code below:

#!/bin/bash
for file in /CSVFiles/Test*.csv # Use file globbing to get the files
do
/*Use a stream editor like awk to print the lines*/
  awk -v FS="," '{
  /* if you want the columns in separate lines */
  print $1;
  print $2;
  }' "$file"
done

Edit 2 : If you need to take the file order into consideration, the code below helps

#!/bin/bash
#Count the total number of files using stat and grep
total=$(stat -f /CSVFiles/Test*.csv | grep -c '^[[:blank:]]*File:')
#User c-style for-loops in bash
for((i=0;i<="$total";i++))
do
awk -v FS="," '{
      /* if you want the columns in separate lines */
      print $1;
      print $2;
      }' "Test${i}.csv" # Feeding the file names in order
done

Upvotes: 1

agc
agc

Reputation: 8406

The output of the OP script seems to be equivalent to this one-liner:

cut -d, -f1,2 /CSVFiles/Test*.csv | tr , '\n'

Upvotes: 0

Vamsi Sudhir
Vamsi Sudhir

Reputation: 11

How about this.

IFS=","
FILES=/CSVFiles/*
cat $FILES|while read Column1 Column2
do
echo $Column1
echo $Column2
done

Upvotes: 1

Related Questions