CODEWITHSUNDEEP
bashunixwhile-loopline-by-line
Sam Thompson
Sam Thompson

Reputation: 1

Reading in File line by line w/ Bash

I'm creating a bash script to read a file in line by line, that is formatted later to be organised by name and then date. I cannot see why this code isn't working at this time though no errors show up even though I have tried with the input and output filename variables on their own, with a directory finder and export command.

export inputfilename="employees.txt"
export outputfilename="branch.txt"
directoryinput=$(find -name $inputfilename)
directoryoutput=$(find -name $outputfilename)
n=1

if [[ -f "$directoryinput" ]]; then
     while read line; do
         echo "$line"
         n=$((n+1))
     done < "$directoryoutput"
 else
    echo "Input file does not exist. Please create a employees.txt file"
 fi

All help is very much appreciated, thank you! NOTE: As people noticed, I forgot to add in the $ sign on the data transfer to file, but it was just in copying my code, I do have the $ sign in my actual application and still no result

Upvotes: 0

Views: 4685

Answers (1)

KamilCuk
KamilCuk

Reputation: 141881

Reading in File line by line w/ Bash

The best and idiomatic way to read file line by line is to do:

while IFS= read -r line; do
  // parse line
  printf "%s" "$line"
done < "file"

More on this topic can be found on bashfaq

However don't read files in bash line by line. You can (ok, almost) always not read a stream line by line in bash. Reading a file line by line in bash is extremely slow and shouldn't be done. For simple cases all the unix tools with the help of xargs or parallel can be used, for more complicated awk and datamesh are used.

done < "directoryoutput"

The code is not working, because you are passing to your while read loop as input to standard input the content of a file named directoryoutput. As such a file does not exists, your script fails.

directoryoutput=$(find -name $outputfilename)

One can simply append the variable value with newline appended to a read while loop using a HERE-string construction:

done <<< "$directoryoutput"

directoryinput=$(find -name $inputfilename)
if [[ -f "$directoryinput" ]]

This is ok as long as you have only one file named $inputfilename in your directory. Also it makes no sense to find a file and then check for it's existance. In case of more files, find return a newline separated list of names. However a small check if [ "$(printf "$directoryinput" | wc -l)" -eq 1 ] or using find -name $inputfilename | head -n1 I think would be better.

while read line;
   do
      echo "$line"
      n=$((n+1))
  done < "directoryoutput"

The intention is pretty clear here. This is just:

 n=$(<directoryoutput wc -l)
 cat "directoryoutput"

Except that while read line removed trailing and leading newlines and is IFS dependent.

Also always remember to quote your variables unless you have a reason not to.

Have a look at shellcheck which can find most common mistakes in scripts.

I would do it more like this:

inputfilename="employees.txt"
outputfilename="branch.txt"

directoryinput=$(find . -name "$inputfilename")
directoryinput_cnt=$(printf "%s\n" "$directoryinput" | wc -l)
if [ "$directoryinput_cnt" -eq 0 ]; then
   echo "Input file does not exist. Please create a '$inputfilename' file" >&2
   exit 1
elif [ "$directoryinput_cnt" -gt 1 ]; then
   echo "Multiple file named '$inputfilename' exists in the current path" >&2
   exit 1
fi

directoryoutput=$(find . -name "$outputfilename")
directoryoutput_cnt=$(printf "%s\n" "$directoryoutput" | wc -l)

if [ "$directoryoutput_cnt" -eq 0 ]; then 
    echo "Input file does not exist. Please create a '$outputfilename' file" >&2
    exit 1
elif [ "$directoryoutput_cnt" -gt 1 ]; then 
   echo "Multiple file named '$outputfilename' exists in the current path" >&2
    exit 1
fi

cat "$directoryoutput"
n=$(<"$directoryoutput" wc -l)

Upvotes: 1

Related Questions