How to output colored text to stdout while also appending the text uncolored to a file?

Question

I wrote a logger function that is supposed to output colored text to screen while also redirecting the text to a log file using the tee command.

This the logger function:

function logger(){
    GREEN=$(tput setaf 2)
    YELLOW=$(tput setaf 3)
    BOLD=$(tput bold)
    UNDERLINE=$(tput smul)
    NOCOLOR=$(tput sgr0)

    case "$1" in
        y)
        echo -e -n "$(timestamp) ${YELLOW}$2 ${NOCOLOR}\n" | tee -a $log_file
        ;;
        g)
        echo -e -n "$(timestamp) ${GREEN}$2 ${NOCOLOR}\n" | tee -a $log_file
        ;;
        b)
        echo -e -n "$(timestamp) ${BOLD}$2 ${NOCOLOR}\n" | tee -a $log_file
        ;;
        u)
        echo -e -n "$(timestamp) ${UNDERLINE}$2 ${NOCOLOR}\n" | tee -a $log_file
        ;;
        n)
        echo -e -n "$(timestamp) $2\n" | tee -a $log_file
        ;;
        *)
        echo "Unknown color!"
        ;;
    esac
}

The problem is that when the text is written in the log file, the color codes are also applied and it makes the text seem dirty.

This is how it looks:

2021-08-31 12:36:41 UTC +0000 ^[[1m------------ Now working on account CompanyEUResearchAndDevelopmentJenkinsSlave in region us-east-2 ^[(B^[[m
2021-08-31 12:36:42 UTC +0000 ^[[32mThe following IPs are allocated and associated: ^[(B^[[m
2021-08-31 12:36:42 UTC +0000 13.11.131.202 18.11.34.219 3.11.227.231
2021-08-31 12:36:42 UTC +0000 ----------------------------------------------------------
2021-08-31 12:36:44 UTC +0000 ^[[33mFound unassociated elastic ips: ^[(B^[[m
2021-08-31 12:36:44 UTC +0000 18.11.91.21
2021-08-31 12:36:45 UTC +0000 ^[[33mIP 18.11.91.21 found in groups: ^[(B^[[m
2021-08-31 12:36:45 UTC +0000 sg-04b6da1d06783ffbf
2021-08-31 12:36:46 UTC +0000 ^[[32mThe rule containing ip 18.11.91.21 has been deleted from security group sg-04b6da1d06783ffbf successfully ^[(B^[[m
2021-08-31 12:36:46 UTC +0000 ^[[32mReleasing ip 18.11.91.21 with AllocationId eipalloc-07528d88e8794c6db ^[(B^[[m
2021-08-31 12:36:46 UTC +0000 ^[[32mAllocation released successfully! ^[(B^[[m
2021-08-31 12:36:47 UTC +0000 ^[[33mFound unassociated elastic ips: ^[(B^[[m
2021-08-31 12:36:47 UTC +0000 3.11.248.103
2021-08-31 12:36:48 UTC +0000 ^[[33mIP 3.11.248.103 found in groups: ^[(B^[[m
2021-08-31 12:36:48 UTC +0000 sg-04b6da1d06783ffbf
2021-08-31 12:36:49 UTC +0000 ^[[32mThe rule containing ip 3.11.248.103 has been deleted from security group sg-04b6da1d06783ffbf successfully ^[(B^[[m
2021-08-31 12:36:49 UTC +0000 ^[[32mReleasing ip 3.11.248.103 with AllocationId eipalloc-0a699e32f4ac844dc ^[(B^[[m
2021-08-31 12:36:49 UTC +0000 ^[[32mAllocation released successfully! ^[(B^[[m
2021-08-31 12:36:49 UTC +0000 End of run

This is how use the function:

logger g "No elastic ips found in account ${role_name}"

I wonder what's the "most right" way to do that?

Answer 1

A modification of Vaphell's answer (towards bottom of page) that removes the color codes from the tee stream before appending/writing to $log_file:

Change this:

echo -e -n "$(timestamp) ${GREEN}$2 ${NOCOLOR}\n" | tee -a $log_file

To this:

echo -e -n "$(timestamp) ${GREEN}$2 ${NOCOLOR}\n" | tee >(sed 's/\x1B[\[\(][0-9;]*[BJKmsu]//g' >> $log_file)

---

Since you're likely looking to call logger() quite often, performance is going to be a big issue, and the biggest improvements are going to come from eliminating as many (unnecessary) subprocess calls as possible.

tl;dr

• use 2x separate echo calls, one to stdout, one to $log_file
• eliminate the subprocess $(timestamp) call which I'm assuming includes a further subprocess call to date
• load the color variables (ie, tput subprocess calls) just once, ie, eliminate the repeated 5x subprocess calls (to tput) on each call to logger()
• for more details continue reading ...

use 2x separate echo calls

per user1934428's comment ... calling echo two times, once to stdout, once to (append to) $log_file is another option, eg:

Instead of this:

echo -e -n "$(timestamp) ${GREEN}$2 ${NOCOLOR}\n" | tee >(sed 's/\x1B[\[\(][0-9;]*[BJKmsu]//g' >> $log_file)

Use this:

echo -e -n "$(timestamp) ${GREEN}$2 ${NOCOLOR}\n"
echo -e -n "$2\n" >> $log_file

Using the following test:

time for i in {1..100}
do
    mylogger g hello > /dev/null
done

NOTES:

• on my system there is a /usr/bin/logger so I've named my function mylogger()
• I've modified my env to only load the color variables once, instead of OP's current method of making 5x tput calls each time the function is called (otherwise the timings for the following tests would be even worse)

Results:

real    0m8.739s       # echo ... | tee >(sed ... >> $log_file)
user    0m3.448s
sys     0m4.706s

real    0m0.105s       # echo ... ; echo ...
user    0m0.047s
sys     0m0.047s

As you can see, the use of two separate echo commands is going to be a LOT faster than the | tee >(sed ...) solution ... or any other solution that requires making (unnecessary) subprocess calls

eliminate the subprocess $(timestamp) (and date) call

I'm assuming the $(timestamp) call is a custom function/binary that generates a date/time string from the current date; if running bash 4.2 (or better) consider using printf to generate the date/time string and store in a local variable, eg:

printf -v NOW '%(%F %H:%M:%S %Z %z)T' -1

NOTES:

• this does not require a subprocess call to date and therefore is going to run quite a bit faster
• see this SE answer for more details
• the general idea would be to make the printf call directly in the logger() function thus eliminating the $(timestamp) subprocess call

load the color variables (ie, tput subprocess calls) just once

As I mentioned in one of my comments, consider populating the color variables once thus eliminating the current process of making 5x subprocess calls to tput each time the logger() function is called; a very simple solution that continues to use OP's current <color>=$(tput ...) format:

function load_colors() {

    if [[ -z "${NOCOLOR}" ]]
    then
        export GREEN=$(tput setaf 2)
        export YELLOW=$(tput setaf 3)
        export BOLD=$(tput bold)
        export UNDERLINE=$(tput smul)
        export NOCOLOR=$(tput sgr0)
    
    fi
}

function logger() {

    load_colors

    case "$1" in
    ... snip ...
    esac
}

Answer 2

Try piping before tee by the command ansi2txt.

For example:

echo -e -n "$(timestamp) ${UNDERLINE}$2 ${NOCOLOR}\n" | ansi2txt | tee -a $log_file