CODEWITHSUNDEEP
bashcsvawk
Shantanu Bedajna
Shantanu Bedajna

Reputation: 581

awk multiple csv file skip headers

I am concatenating multiple csv files with awk and modifying the rows but i do not want the headers to print i am taking care of the headers in another method i just want awk to skip the headers

here is my code

OutFileName="Final.csv"                       # Fix the output name
i=0                                       # Reset a counter
for filename in ./*.csv; do 
 if [ "$filename"  != "$OutFileName" ] ;      # Avoid recursion 
 then 
   if [[ $i -eq 0 ]] ; then 
      head -1  $filename >   $OutFileName # Copy header if it is the first file
   fi
   text=$(echo $filename | cut -d '_' -f 2-) #spilliting the string to get the date 
   text=$(echo $text | cut -d '.' -f -1)
   awk -F',' -v txt="$text" 'FNR>1||$0=txt" "$1 FS $0' OFS=',' $filename | cat >> $OutFileName # cocatinating with awk 
   i=$(( $i + 1 )) #increse number
 fi
done

it gives an output like this

Trialtime   type    Track   time    Notes   Athlete 
Trialtime   type    Track   time    Notes   Athlete 
20170101 07:15:00   07:15:00    Warmup  ABC 85.2    Initial warmup  Jon
20170101 07:45:00   07:45:00    Sprint1 ABC 59.44   First Sprint    Jon
20170101 08:30:00   08:30:00    TRIAL   ABC 57.21   Final attempt   Jon
20170101 08:00:00   08:00:00    Warmup  ABC 120.51  Initial warmup  Bill
20170101 08:40:05   08:40:05    Sprint1 ABC 61.35   First Sprint    Bill
20170101 09:15:00   09:15:00    Sprint2 ABC 60.08   Second Sprint   Bill
20170101 10:30:00   10:30:00    TRIAL   ABC 60.37   Final attempt   Bill
20170101 07:15:00   07:15:00    Warmup  ABC 85.2    Initial warmup  Jon
20170101 07:45:00   07:45:00    Sprint1 ABC 59.44   First Sprint    Jon
20170101 08:30:00   08:30:00    TRIAL   ABC 57.21   Final attempt   Jon
20170101 08:00:00   08:00:00    Warmup  ABC 120.51  Initial warmup  Bill
20170101 08:40:05   08:40:05    Sprint1 ABC 61.35   First Sprint    Bill
20170101 09:15:00   09:15:00    Sprint2 ABC 60.08   Second Sprint   Bill
20170101 10:30:00   10:30:00    TRIAL   ABC 60.37   Final attempt   Bill
Trialtime   type    Track   time    Notes   Athlete 
201701023 07:15:00  07:15:00    Warmup  ABC 85.2    Initial warmup  Jon
201701023 07:45:00  07:45:00    Sprint1 ABC 59.44   First Sprint    Jon
201701023 08:30:00  08:30:00    TRIAL   ABC 57.21   Final attempt   Jon
201701023 08:00:00  08:00:00    Warmup  ABC 120.51  Initial warmup  Bill
201701023 08:40:05  08:40:05    Sprint1 ABC 61.35   First Sprint    Bill
201701023 09:15:00  09:15:00    Sprint2 ABC 60.08   Second Sprint   Bill
201701023 10:30:00  10:30:00    TRIAL   ABC 60.37   Final attempt   Bill
201701023 07:15:00  07:15:00    Warmup  ABC 85.2    Initial warmup  Jon
201701023 07:45:00  07:45:00    Sprint1 ABC 59.44   First Sprint    Jon
201701023 08:30:00  08:30:00    TRIAL   ABC 57.21   Final attempt   Jon
201701023 08:00:00  08:00:00    Warmup  ABC 120.51  Initial warmup  Bill
201701023 08:40:05  08:40:05    Sprint1 ABC 61.35   First Sprint    Bill
201701023 09:15:00  09:15:00    Sprint2 ABC 60.08   Second Sprint   Bill
201701023 10:30:00  10:30:00    TRIAL   ABC 60.37   Final attempt   Bill

you can see awk is taking the headers each time for each csv file is there a way to ignore the headers ?

Upvotes: 0

Views: 1442

Answers (2)

Nahuel Fouilleul
Nahuel Fouilleul

Reputation: 19315

changing || with {} and using explicit print

awk -F',' -v txt="$text" 'FNR>1 { print txt" "$1 FS $0 }'

Upvotes: 2

Marcos Oliveira
Marcos Oliveira

Reputation: 509

In your code, just add NR!=1 &&:

OutFileName="Final.csv"                       # Fix the output name
i=0                                       # Reset a counter
for filename in ./*.csv; do 
 if [ "$filename"  != "$OutFileName" ] ;      # Avoid recursion 
 then 
   if [[ $i -eq 0 ]] ; then 
      head -1  $filename >   $OutFileName # Copy header if it is the first file
   fi
   text=$(echo $filename | cut -d '_' -f 2-) #spilliting the string to get the date 
   text=$(echo $text | cut -d '.' -f -1)
   awk -F',' -v txt="$text" 'FNR>1||NR!=1 && $0=txt" "$1 FS $0' OFS=',' $filename | cat >> $OutFileName # cocatinating with awk 
   i=$(( $i + 1 )) #increse number
 fi
done

This will skip the first line.

Upvotes: 1

Related Questions