CODEWITHSUNDEEP
pythonloggingpython-2.7
introiboad
introiboad

Reputation: 1017

logging to a file and stderr with a Python logger

I am trying to add a module to a project that uses the standard Python (2.7) logging libraries. What I want to achieve is the following:

Please note that the key issue here is that I do not want to modify the global logging settings, since those are controlled by the main app. Otherwise I would use the example in here:

http://docs.python.org/2/howto/logging-cookbook.html#logging-to-multiple-destinations

I've tried this:

logger = logging.getLogger('my_module')
logger.setLevel(logging.WARNING)
fh = logging.FileHandler('my_module.log')
fh.setLevel(logging.DEBUG)
logger.addHandler(fh)

but then I only get warning messages in my file.

If I replace the above line with:

logger.setLevel(logging.DEBUG)

then I get debug messages in both stderr and my file.

Then I tried this:

logger = logging.getLogger('my_module')
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.NullHandler())
console = logging.StreamHandler()
console.setLevel(logging.WARNING)
logger.addHandler(console)
fh = logging.FileHandler('my_module.log')
fh.setLevel(logging.DEBUG)
logger.addHandler(fh)

But again no luck.

So can I achieve this without having to call basicConfig() or anything else that affects the global logging infrastructure? I only want to modify my own module here if possible.

Thanks!

Upvotes: 4

Views: 3819

Answers (2)

jfs
jfs

Reputation: 414205

  • Anything equal or worse than WARNING goes to wherever the main program (outside of my control) has configured the logging to go (usually stderr)
  • Everything else (debug messages mainly) go to a log file

You can't do it with a single logger without setting WARNING level on ancestor's handlers. From the docs:

Logger.propagate

If this evaluates to true, logging messages are passed by this logger and by its child loggers to the handlers of higher level (ancestor) loggers. Messages are passed directly to the ancestor loggers’ handlers - neither the level nor filters of the ancestor loggers in question are considered.

what do you suggest a module can do in this situation?

In general other than the setting of WARNING level for the logger and adding NullHandler(), library code shouldn't configure logging at all.

If you want an easy way for the main application to provide you with a debug info then define a function that configures DEBUG level logging for your library.

I don't want the library to be DEBUG except for the file, where I would like to see everything.

If you can't control the main program then you shouldn't create debug files as a side-effect of using your_module unless explicitly asked to do so:

import your_module # in the main application

your_module.get_logger().log_to_file(filename) # without this line your module
                                               # shouldn't create debug files
...
your_module.some_function()

I am not sure how this logger hierarchy works: the logger I created is a child of the root logger that the app configured?

'' is a root logger. 'a.b' is a child of 'a' logger.

Upvotes: 2

Jay Elrod
Jay Elrod

Reputation: 738

Try creating two loggers pointing to the same file. One .ERROR and one .WARNING. Never done this with Python, but I don't see why it wouldn't work. As long as you flush your streams after writes, they should not interfere with one another, unless you are running co-routines, in which case you would need to program in some locks and such. As long as your program runs linearly you shouldn't have any worries.

Upvotes: 0

Related Questions