CODEWITHSUNDEEP
pythonbashloggingstdout
JStroop
JStroop

Reputation: 485

Print python stdout to stdout

This is a little tricky to explain, so I apologize in advance.

I have a command line application written in Python. It logs to stderr and stdout; the logging filters are configured as follows:

class StdErrFilter(logging.Filter):
    def filter(self,record):
        return 1 if record.levelno >= 30 else 0

class StdOutFilter(logging.Filter):
    def filter(self,record):
        return 1 if record.levelno <= 20 and record.levelno > 10 else 0

and the function that sets up logging looks like this:

def setup_logging(logger_name):
    logger = logging.getLogger(logger_name)
    logger.setLevel(logging.DEBUG)

    formatter = logging.Formatter(fmt=LOG_FMT)

    err_handler = logging.StreamHandler(sys.__stderr__)
    err_handler.addFilter(StdErrFilter())
    err_handler.setFormatter(formatter)
    logger.addHandler(err_handler)

    out_handler = logging.StreamHandler(sys.__stdout__)
    out_handler.addFilter(StdOutFilter())
    out_handler.setFormatter(formatter)
    logger.addHandler(out_handler)

    return logger

When the application is invoked directly it works as I'd expect, i.e. info() and below print to stdout, and warn() and above print to stderr. However, when the exact same call is made inside a bash script, info and below don't print to the terminal, but stderr still works as expected. I can make the script pipe stdout to a file, but not to the terminal. Any ideas what's going on?

EDIT I should have shown the bash script as well. There's some other logic related to the building the args for the Python app in there, but the actual call is near the bottom in the for loop. Note that the Python app is an executable module.

#!/bin/bash

# provide one argument, the directory to search for ".xml" files.
dir_to_index=$1

FIND=/usr/bin/find
PYTHON=/usr/bin/python

SOLR="http://localhost:8983/solr/pul-development" 

BIN="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

marc_to_solr=$BIN/../marc_to_solr
PYTHONPATH=$marc_to_solr:$PYTHONPATH

db_config=$BIN/../etc/database.ini
config=db_config=$BIN/../etc/config.ini

for f in `$FIND $dir_to_index -name "*.xml"`; do
    echo "File: $f"
    cmd="$PYTHON -m marc_to_solr -d $db_config -s $SOLR -c etc/config.ini -i $f"
    `$cmd`
done

Upvotes: 1

Views: 3060

Answers (2)

Dan Lecocq
Dan Lecocq

Reputation: 3493

I believe the problem lies in the invocation of your python script. By placing $cmd within the ticks, you're telling bash to run the stdout output of the program as a bash command. This is the simplest case I could contrive

echo 'import sys
sys.stdout.write(str(1))
sys.stderr.write(str(2))' > test.py

cmd='python test.py'

And now we can try invoking it a couple different ways

# If you invoke it like this, it works fine
$cmd
# And this will work, too
bash -c "$cmd"
# If you invoke it like this, you'll get some complaints
`$cmd`

Upvotes: 2

bit4
bit4

Reputation: 176

The problem is in the line

`$cmd`

Backticks in bash mean to redirect stdout to an internal bash buffer that you can include in another command or assign to a bash variable. Just omit the backticks and it will do what you expect.

Upvotes: 2

Related Questions