CODEWITHSUNDEEP
bash
Uwe Geuder
Uwe Geuder

Reputation: 2317

Writing a Unix filter using bash

If a Unix/Linux command accepts its input data from the standard input and produces its output (result) on standard output is known as a filter.

The trivial filter is cat. It just copies stdin to stdout without any modification whatsoever.

How do I implement cat in bash? (neglecting the case that the command gets command line arguments)

I came up with

#! /bin/bash

while IFS="" read -r line
do
  echo -E "$line"
done

That seems to work in most cases, also for text files containing some binary bytes as long as they are not null bytes. However, if the last line does not end in a newline character, it will be missing from the output.

How can that be fixed?

I'm nearly sure this must have been answered before, but my searching skills don't seem to be good enough.

Obviously I don't want to re-implement cat in bash: It wouldn't work anyway because of the null byte problem. But I want to extend the basic loop to do some custom processing to certain lines of a text file. However, we have all seen text files without the final line feed, so I'd prefer if that case could be handled.

Upvotes: 4

Views: 1070

Answers (1)

chepner
chepner

Reputation: 530922

Assuming you don't need to work with arbitrary binary files (since the shell cannot store null bytes in a variable), you can handle a file that isn't terminated by a newline by checking if line is not empty once the loop exits.

while IFS= read -r line; do
    printf '%s\n' "$line"
done
if [ -n "$line" ]; then
    printf '%s' "$line"
fi

In the loop, we output the newline that read stripped off. In the final if, we don't output a newline because $line would be empty if the last line read by the while loop had ended with one.

Upvotes: 5

Related Questions