CODEWITHSUNDEEP
linuxbashawk
Amit
Amit

Reputation: 57

AWK - syntax error: unexpected end of file

I have about 300 files in a folder, trying to remove comma in CSV, when I run in the loop I got an error

MY CODE :

#!/bin/bash
FILES=/home/whoisdat/all-data/*
{

for f in $FILES
do  

{
awk -F'"' -v OFS='' '{ for (i=2; i<=NF; i+=2) gsub(",", "", $i) } 1' $f > allan-$f
}
done

Error:

[email protected] [all-data]# sh /home/all-data/unique.sh
/home/whoisdat/all-data/unique.sh: line 12: syntax error: unexpected end of file

Upvotes: 0

Views: 320

Answers (2)

Ed Morton
Ed Morton

Reputation: 203684

The right way to do what you're doing is:

awk -F'"' -v OFS='' '
    FNR==1 { close(out); out="allan-"FILENAME }
    { for (i=2; i<=NF; i+=2) gsub(/,/, "", $i); print > out }
' /home/whoisdat/all-data/*

We close the previous output file when we start reading the next input file to avoid a "too many open files" error from most awks when we get past a limit of a dozen or so (GNU awk which can handle many open files suffers a slowdown instead which is also undesirable), and we only close it there instead of once per input line processed to avoid the significant performance hit of opening and closing output files once per input line.

The above assumes you aren't running the command under /home/whoisdat/all-data/ and so creating allan-* files in /home/whoisdat/all-data/ while the script is running.

Upvotes: 0

Raman Sailopal
Raman Sailopal

Reputation: 12877

awk -F'"' -v OFS='' '{ for (i=2; i<=NF; i+=2) gsub(",", "", $i);print $0 >> ("allan-"FILENAME);close("allan-"FILENAME) }' /home/whoisdat/all-data/*

There is no need to loop on the files, just allow awk to process all the files and use FILENAME to track the files being processed.

Upvotes: 1

Related Questions