CODEWITHSUNDEEP
bashsed
Jai
Jai

Reputation: 375

Ignoring First line/Column header while reading a file in bash

I am trying to read from a source txt file in bash and I want to ignore the first line which is the column.After searching around one solution was to use "sed" with my while loop like below :

#!/bin/bash
filename="source2.txt"
#fp=`sed 1d source2.txt`
#echo $fp
sed 1d $filename | while IFS=, read -r accountId ProductId Product
do 
echo "Account $accountId has productId $ProductId and product $Product"
done < $filename

But the sed command does not seem to work.Keeps giving all the contents with header.I tried adding double quotes to 1d and also $filename but does not work.

Here is my sample input file content

AccountId ProductId Product
300100051205280,300100051161910,content1
300100051199355,300100051161876,content2

I am using Editra editor for creating my bash script.Can anyone help me why this is not working.Thanks for the help in advance.

Upvotes: 27

Views: 64127

Answers (3)

mgutt
mgutt

Reputation: 6177

Another variant which skips the first iteration by checking the existance of a variable:

while IFS=, read -r accountId ProductId Product; do
  # skip header
  if [[ ! $first_line ]]; then
    first_line=1
    continue
  fi
  # process
done < $filename

Or skip the first line with tail -n +2 (which is technically the same as sed 1d):

while IFS=, read -r accountId ProductId Product; do
  # process 
done < <(tail -n +2 $filename)

Upvotes: 1

Kyler Middleton
Kyler Middleton

Reputation: 136

Other answers are much more technically savvy, but I have a simple hack that works quite well, as long as the header's first title isn't changed.

while IFS=, read -r accountId ProductId Product
do 
  # If accountId matches Header line value, continue
  if [[ $accountId == "AccountId" ]]; then
     continue
  fi
  #Do your other things, assume all values are real
done < $filename

Upvotes: 10

chepner
chepner

Reputation: 531625

Use an extra read inside a compound command. This is more efficient than using a separate process to skip the first line, and prevents the while loop from running in a subshell (which might be important if you try to set any variables in the body of the loop).

{
    read
    while IFS=, read -r accountId ProductId Product
    do 
        echo "Account $accountId has productId $ProductId and product $Product"
    done
} < $filename

--

The problem with your original attempt is that you were providing two sources of input to the while loop (via the pipe from sed, and via an input reduction). Dropping the input redirection would fix that.

sed 1d $filename | while IFS=, read -r accountId ProductId Product
do 
    echo "Account $accountId has productId $ProductId and product $Product"
done

Upvotes: 68

Related Questions