CODEWITHSUNDEEP
bashfile-descriptor
Jay
Jay

Reputation: 19857

Script works until it's modified to redirect stdout & stderr to a log file

Explanation

When The Script below is ran with no modifications, it outputs (correctly):

two one START

After uncommenting the exec statements in the script below, file descriptor 3 points to standard output, and file descriptor 4 points to standard error:

exec 3>&1 4>&2

And all standard output & standard error gets logged to a file instead of printing to the console:

# Two variances seen in the script below
exec 1>"TEST-${mode:1}.log" 2>&1  # Log Name based on subcommand
exec 1>"TEST-bootstrap.log" 2>&1  # Initial "bootstrap" log file

When the exec statements are in place, the script should create three files (with contents):

TEST-bootstrap.log (sucessfully created - empty file)

TEST-one.log (sucessfully created)

one START

TEST-two.log (is not created)

two one START

But instead it seems to stop after the first log file and never creates TEST-two.log. What could be the issue considering it works with the exec statements commented out?

The Script

#!/bin/bash
SELFD=$(cd -P $(dirname ${0}) >/dev/null 2>&1; pwd)  # Current scripts directory
SELF=$(basename ${0})   # Current scripts name
SELFX=${SELFD}/${SELF}  # The current script exec path

fnOne() { echo "one ${*}"; }
fnTwo() { echo "two ${*}"; }

subcommand() {
    # exec 3>&1 4>&2
    # exec 1>"TEST-${mode:1}.log" 2>&1
    case "${mode}" in
        -one) fnOne   ${*}; ;;
        -two) fnTwo   ${*}; ;;
    esac
}

bootstrap() {
    # exec 3>&1 4>&2
    # exec 1>"TEST-bootstrap.log" 2>&1
    echo "START" \
        | while read line; do ${SELFX} -one ${line}; done \
        | while read line; do ${SELFX} -two ${line}; done
    exit 0
}

mode=${1}; shift

case "${mode:0:1}" in
    -) subcommand ${*} ;;
    *) bootstrap ${mode} ${*} ;;
esac
exit 0

This script is strictly an isolated example of the problem I am facing in another more complex script of mine. I've tried to keep it as concise as possible but will expand upon it if needed.

What I'm trying to accomplish (extra reading for those interested)

In The Script above, I am using while loops instead of for simplicity sake, and am using simple functions that echo "one" and "two" for ease of debugging & asking the question here.

In my actual script, the program would have the following functions: list-archives, download, extract, process that would fetch a list of archives from a URL, download them, extract their contents, and process the resulting files respectively. Considering these operations take a varying amount of time, I planned on running them in parallel. My script's bootstrap() function would look something like this:

# program ./myscript
list-archives "${1}" \
    | parallel myscript -download \
    | parallel myscript -extract \
    | parallel myscript -process

And would be called like this:

./myscript "http://www.example.com"

What I'm trying to accomplish is a way to start a program that can can call it's own functions in parallel and record everything it does. This would be useful to determine when the data was last fetched, debug any errors, etc.

I'd also like to have the program record logs when it's invoked with a subcommand, e.g.

# only list achives
>> ./myscript -list-archives "http://www.example.com"
# download this specific archive
>> ./myscript -download "http://www.example.com/archive.zip" 
# ...

Upvotes: 0

Views: 224

Answers (2)

Jay
Jay

Reputation: 19857

I've accepted @Barmar's answer as the answer and the comment left by @Wumpus Q. Wumbley nudged me in the right direction.

The solution was to use tee /dev/fd/3 on the case statement like so:

subcommand() {
    exec 3>&1 4>&2
    exec 1>"TEST-${mode:1}.log" 2>&1
    case "${mode}" in
        -one) fnOne   ${*}; ;;
        -two) fnTwo   ${*}; ;;
    esac | tee /dev/fd/3
}

Upvotes: 0

Barmar
Barmar

Reputation: 781058

I think this is the problem:

echo "START" \
    | while read line; do ${SELFX} -one ${line}; done \
    | while read line; do ${SELFX} -two ${line}; done

The first while loop reads START from the echo statement. The second while loop processes the output of the first while loop. But since the script is redirecting stdout to a file, nothing is piped to the second loop, so it exits immediately.

I'm not sure how to "fix" this, since it's not clear what you're trying to accomplish. If you want to feed something to the next command in the pipeline, you can echo to FD 3, which you've carefully used to save the original stdout.

Upvotes: 2

Related Questions