CODEWITHSUNDEEP
linuxbash
shay
shay

Reputation: 867

Read from two files in simultaneously in bash when they may be missing trailing newlines

I have two text files that I'm trying to read through line by line at the same time. The files do not necessarily have the same number of lines, and the script should stop reading when it reaches the end of either file. I'd like to keep this as "pure" bash as possible. Most of the solutions I've found for doing this suggest something of the form:

while read -r f1 && read -r f2 <&3; do
    echo "$f1"
    echo "$f2"
done < file1 3<file2

However this fails if the last line of a file does not have a newline.

If I were only reading one file I would do something like:

while IFS='' read -r line || [[ -n "$line" ]]; do
    echo "$line"
done < file1

but when I try to extend this to reading multiple files by doing things like:

while IFS='' read -r f1 || [[ -n "$f1" ]] && read -r f2 <&3 || [[ -n "$f2" ]]; do
    echo "$f1"
    echo "$f2"
done < file1 3<file2

or

while IFS='' read -r f1 && read -r f2 <&3 || [[ -n "$f1" || -n "$f2" ]]; do
    echo "$f1"
    echo "$f2"
done < file1 3<file2

I can't seem to get the logic right and the loop either doesn't terminate or finishes without reading the last line.

I can get the desired behavior using:

while true; do
    read -r f1 <&3 || if [[ -z "$f1" ]]; then break; fi;
    read -r f2 <&4 || if [[ -z "$f2" ]]; then break; fi;
    echo "$f1"
    echo "$f2"
done 3<file1 4<file2

However this doesn't seem to match the normal (?)

while ... read ...; do
     ... 
done

idiom that I see for reading files.

Is there a better way to simultaneously read from two files that might have differing numbers of lines and last lines that are not newline terminated?

What are the potential drawbacks of my way of reading the files?

Upvotes: 3

Views: 67

Answers (1)

Gordon Davisson
Gordon Davisson

Reputation: 125748

You can override the precedence of the && and || operators by grouping them with { }:

while { IFS= read -r f1 || [[ -n "$f1" ]]; } && { IFS= read -r f2 <&3 || [[ -n "$f2" ]]; }; do

Some notes: You can't use ( ) for grouping in this case because that forces its contents to run in a subshell, and variables read in subshells aren't available in the parent shell. Also, you need a ; before each }, so the shell can tell it isn't just a parameter to the last command. Finally, you need IFS= (or equivalently IFS='') before each read command, since assignments given as a prefix to a command apply only to that one command.

Upvotes: 5

Related Questions