CODEWITHSUNDEEP
pythonbash
Gabriel
Gabriel

Reputation: 9442

python stderr gets appended on top while redirecting with bash

I have the following problem:

def main():

    try: 
        # DO some stuff here, load some class and execute it thats all
        processFile = cf.jsonLoad(opts.processFile);
        # load process module/class
        mod, ProcessClass = iH.importClassFromModule( **processFile["processClass"] )
        process = ProcessClass( processFile, jobGenModules = {"importHelpers":iH, "commonFunctions" : cf} )
        process.doProcessing()

    except Exception as e:
        print("====================================================================", file=sys.stderr)
        print("Exception occured here: " + str(e), file=sys.stderr)
        print("====================================================================", file=sys.stderr)
        traceback.print_exc(file=sys.stderr)
        return 1

if __name__ == "__main__":
   sys.exit(main());

when I launch the program from bash by redirecting stderr and stdout to the file by doing

yell() { echo "$0: $*" >&2; }
die() { yell "$*"; cleanup ; exit 111 ; }
try() { "$@" || die "cannot $*"; }
executeProg(){
    python3 main.py >> log.txt 2>&1
}
try executeProg

the stderr output of the exception above gets appended at the beginning of the file?

Is that general behavior of bash? Or am I doing something wrong in python? How can I make that the stderr is also appended at the position where the stdout is, meaning at the end of the file...?

Upvotes: 0

Views: 212

Answers (2)

shanmuga
shanmuga

Reputation: 4499

2>&1 This basically says redirect stderr to stdout.

I tried the following code.

a = 0
b = 1

print("hi")
try:
    c = b/a
except ZeroDivisionError:
    print("ZeroDivisionError")
print("done")

To execute I used python a.py >> a.txt 2>&1. The content of a.txt after this is

hi
ZeroDivisionError
done

This shows whatever is printed first comes first in the file.

Upvotes: 1

secolive
secolive

Reputation: 589

Such behaviour is typically caused by buffering, e.g. the stdout output of your python code "DO STUFF HERE" has not yet been sent to the file. Typically you will not see any problem in the console, but the "problem" appears once you redirect to a file.

You might try to manually flush stdout before outputting the exception details, as follows:

except Exception as e:
    sys.stdout.flush()
    print(...

You could also try disabling buffering altogether, see Disable output buffering

Upvotes: 2

Related Questions