CODEWITHSUNDEEP
csvawkcsh
astro_andy
astro_andy

Reputation: 37

Reading a csv file into csh using awk

Suppose I have a comma-separated file called 'list.txt' which contains the following:

1,fileA
2,fileB

I want to read these into a while loop in my csh script, so that I can manipulate the comma separated fields separately. I need to scale this up for any number of lines in the input text file as opposed to the 2 in this example.

    #!/bin/csh
    set j=1
    while ($j <= 2)
      set index = "`awk -F"," '{if (NR==$j) print $1}' list.txt`"
      set file = "`awk -F"," '{if (NR==$j) print $2}' list.txt`"
      echo $index
      echo $file
      @ j++
    end

So I would expect the output of this to be

 1
 fileA
 2
 fileB

but in stead I get:

1,fileA
1,fileA
2,fileB
2,fileB

What am I missing here? If I run an equivalent awk command for any given line in the terminal outside of my csh script, it works as I expect it to.

awk -F"," '{if (NR==1) print $1}' list.txt

returns

1

I believe in the csh script the trouble is to do with the double inverted commas specifying the delimiter, which means something else in csh, but I can't figure out the solution.

Upvotes: 0

Views: 1710

Answers (4)

Ed Morton
Ed Morton

Reputation: 203995

Here's how to do what you've told us about so far using any awk and a shell that supports arrays, e.g. bash:

$ awk -F, -v OFS='\n' '{$1=$1}1' file
1
fileA
2
fileB

$ IFS=$'\n' arr=( $(awk -F, -v OFS='\n' '{$1=$1}1' file) )

$ echo "${arr[0]}"
1
$ echo "${arr[1]}"
fileA
$ echo "${arr[2]}"
2
$ echo "${arr[3]}"
fileB

Now, if you tell us what it is you want to DO with the awk output we can provide guidance.

To be clear, the echos above are NOT part of any proposed solution, they are just to show an array was populated. Here's the same in a loop if you prefer (I added some surrounding < > chars in the echos just to make it clear what the echos are doing vs the awk):

$ cat tst.sh
IFS=$'\n' arr=( $(awk -F, -v OFS='\n' '{$1=$1}1' file) )
for i in "${arr[@]}"
do
    echo "< $i >"
done

$ ./tst.sh
< 1 >
< fileA >
< 2 >
< fileB >

but it's not clear any of that is useful. This is much more likely to be what you want instead:

$ awk -F, -v OFS='\n' '{$1=$1}1' file | xargs -I {} -n1 echo "<" {} ">"
< 1 >
< fileA >
< 2 >
< fileB >

Again, until you show us what you want to do with the contents of your input file, we can't help you figure out how to implement a solution.

Given what you just stated in a comment about wanting to run a command named gausmooth with args that include file names generated from your input file, here's one way to do that (remove the echo to execute the command instead of just printing it):

$ cat tst.sh
awk -F, '{print $2$1}' file |
xargs -I {} echo gausmooth in="/home/Documents/{}" out="/home/Documents/gs_{}"

$ ./tst.sh
gausmooth in=/home/Documents/fileA1 out=/home/Documents/gs_fileA1
gausmooth in=/home/Documents/fileB2 out=/home/Documents/gs_fileB2

Again - if that's not what you want, tell us the rest of the story.

Given your new, new requirements this may be what you want:

$ cat file
fileA1,fileA2
fileB1,fileB2
$
$ while IFS=, read -r in1 in2
do
    echo gausmooth in1="/home/Documents/$in1" in2="/home/Documents/$in2" out="/home/Documents/gs_${in1%%[0-9]*}"
done < file
gausmooth in1=/home/Documents/fileA1 in2=/home/Documents/fileA2 out=/home/Documents/gs_fileA
gausmooth in1=/home/Documents/fileB1 in2=/home/Documents/fileB2 out=/home/Documents/gs_fileB

As I mentioned in a different comment the usual advice is to avoid shell loops in general but I use them sometimes for simplicity/clarity when I have control over the input and it is easily constrained so that's what I've done in this case. The assumptions are that none of your file names contain commas or newlines and that the name of the out file can be created by stripping trailing digits off the first in file.

Upvotes: 0

astro_andy
astro_andy

Reputation: 37

The solution in this case, was to use cut in stead of awk.

#!/bin/csh
foreach LINE ( `cat list.txt` )
  set index = `echo "$LINE" | cut -d',' -f 1`
  set file  = `echo "$LINE" | cut -d',' -f 2`
  echo $index
  echo $file
end

which produces the desired output

1
fileA
2
fileB

and the strings are stored in variables for further use.

Upvotes: -1

Akshay Hegde
Akshay Hegde

Reputation: 16997

Try, if your interest is awk

[akshay@localhost tmp]$ cat infile
1,fileA
2,fileB

[akshay@localhost tmp]$ awk -F, '{$1=$1}1' OFS="\n" infile
1
fileA
2
fileB

[akshay@localhost tmp]$ awk 'gsub(/,/,"\n")+1' infile
1
fileA
2
fileB

Upvotes: 1

Jotne
Jotne

Reputation: 41460

Not sure what you goal is, but you can do:

tr ',' '\n' <file
1
fileA
2
fileB

Upvotes: 1

Related Questions