CODEWITHSUNDEEP
pythonloggingerror-logging
Buggy B
Buggy B

Reputation: 765

Python: Logging actions and exceptions with time stamp

I am writing a script that connects to N hosts via SSH ... queries the 3rd party system and extracts data and then displays all the collected data in a certain format.

I want to log all the actions the script is executing as well as any exceptions encountered on to the console and to a log file, so that the user can see what is happening while the script is running (If someone used Ansible - then just like the output we get on the console and logs when running the playbooks)

Expected output

I hope I am able to explain it correctly - Logging actions and exceptions with timestamp for the whole script and all the data iterations.

Please advice and if possible with an example script that uses the technique. Thanks

Upvotes: 2

Views: 6484

Answers (2)

Marcy
Marcy

Reputation: 180

You can have a look here for some more detailed guidance. Here's how I usually set up logging on my stuff:

import logging

...

logger = logging.getLogger()
log_handler = logging.StreamHandler(sys.stdout)
log_handler.setFormatter(logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s - %(funcName)s - line %(lineno)d"))
log_handler.setLevel(logging.DEBUG)
logger.addHandler(log_handler)
logger.setLevel(logging.DEBUG)

This will produce output like this, for any event from DEBUG upwards:

2017-05-16 13:30:03,193 - root - INFO - Starting execution - main - line 35
2017-05-16 13:30:03,206 - root - DEBUG - Config file grabbed successfully - readConfig - line 71
...
2017-05-15 13:30:26,792 - root - WARNING - Reached maximum number of attempts (3) for this request; skipping request. - main - line 79
2017-05-15 13:30:26,797 - root - ERROR - Failed to grab item. Unfortunately, this is a showstopper :( - main - line 79

The above is produced by a line in the main function of my app, that reads:

logger.info("Starting execution")

Another line in my readConfig function:

logging.debug("Config file grabbed successfully")

And another two lines in main again:

logging.warning("Reached maximum number of attempts ({max_attempts}) for this request; skipping request.".format(max_attempts = max_tries))
...
logging.error("Failed to grab item. Unfortunately, this is a showstopper :(")

Then it's a matter of how much information and context you need on each log entry. Have a look here at formatting the entries, and here at formatters. I'll have these sent to me via email, anytime the app runs triggered by crontab, by adding MAILTO = root to the top of my crontab file, and making sure my system email is properly set.

If you want to set it to go to the console and a file, you'll just need to set two different handlers. This answer provides a good example, where you'd set a StreamHandler to log to the console, and a FileHandler to log to a file. So instead of setting it up as I mentioned above I usually do, you could try:

import logging

...

# Set up logging and formatting
logger = logging.getLogger()
logFormatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s - %(funcName)s - line %(lineno)d")

# Set up the console handler
consoleHandler = logging.StreamHandler()
consoleHandler.setFormatter(logFormatter)
logger.addHandler(consoleHandler)

# Set up the file handler 
fileHandler = logging.FileHandler("{0}/{1}.log".format(logPath, fileName))
fileHandler.setFormatter(logFormatter)
logger.addHandler(fileHandler)

# Set up logging levels
consoleHandler.setLevel(logging.DEBUG)
fileHandler.setLevel(logging.DEBUG)
logger.setLevel(logging.DEBUG)

Upvotes: 3

James Danylik
James Danylik

Reputation: 371

Check out the logging module here, there's a nice example section with both basic and advanced applications. Doing stuff in the format you've described appears to be included in the tutorial.

Upvotes: 1

Related Questions