CODEWITHSUNDEEP
pythonjsonpython-3.xlogging
dpatino
dpatino

Reputation: 109

Fix too many logs using json config file in python

I've a python application which is getting bigger day a day and now I'm trying to change its logging system. Thus, I've prepared a json config file for logging properties. Like this:

{
    "version": 1,
    "disable_existing_loggers": false,
    "formatters": {
        "simple": {
            "format": "%(asctime)s - %(name)s - %(levelname)s - %(message)s",
            "datefmt": "%Y-%m-%d %H:%M:%S"
        }
    },

    "handlers": {
        "rotate_timed": {
            "level": "DEBUG",
            "class": "logging.handlers.TimedRotatingFileHandler",
            "formatter": "simple",
            "when": "midnight",
            "backupCount": 5,
            "filename": "/var/log/v4m_agentd.log",
            "encoding": "utf8"
        }
    },

    "loggers": {
        "my_module": {
            "level": "DEBUG",
            "handlers": ["rotate_timed"]
        }
    },

    "root": {
        "level": "DEBUG",
        "handlers": ["rotate_timed"],
        "propagate": false
    }
}

My main program get logger properties using:

logging_config_json_file = open(os.path.join(os.path.dirname(os.path.realpath(__file__)), "logging.json"))
parsed_logging_data = json.load(logging_config_json_file)
logging.config.dictConfig(parsed_logging_data)

My modules call the logger like:

class StoreData(object):
    def __init__(self):
        self.logger = logging.getLogger(__name__)

Since I've changed logging method, to log using a config file, I've a lot of log lines in my .log file, due to all imported modules in my classes (like: apscheduler, werkzeug, ...) are using this config file to log in their messages in my log file and this doesn't happend before.

Before:

2015-04-09 10:29:18 - Agent Get Data - DEBUG - GETTING DATA
2015-04-09 10:29:18 - Prepare DB Data - DEBUG - -----Init prepare data for DB-----
2015-04-09 10:29:21 - Jobs Listener - INFO - The job 'Get Data' worked :)       Next run at: 2015-04-09 10:29:23

After:

2015-04-10 10:27:59 - Agent Get Data - DEBUG - GETTING DATA
2015-04-10 10:27:59 - Prepare DB Data - DEBUG - -----Init prepare data for DB-----
2015-04-10 10:28:02 - apscheduler.executors.default - INFO - Job "Get Data (trigger: interval[0:00:05], next run at: 2015-04-10 10:28:04 CEST)" executed successfully
2015-04-10 10:28:02 - Jobs Listener - INFO - The job 'Get Data' worked :)       Next run at: 2015-04-10 10:28:04
2015-04-10 10:28:04 - apscheduler.scheduler - DEBUG - Looking for jobs to run
2015-04-10 10:28:04 - apscheduler.scheduler - DEBUG - Next wakeup is due at 2015-04-10 10:28:09.847414+02:00 (in 4.998605 seconds)
2015-04-10 10:28:04 - apscheduler.executors.default - INFO - Running job "Get Data (trigger: interval[0:00:05], next run at: 2015-04-10 10:28:09 CEST)" (scheduled at 2015-04-10 10:28:04.847414+02:00)

I only want to write my own log messages. How could I fix it?

Thanks a lot!

Upvotes: 1

Views: 827

Answers (2)

bruno desthuilliers
bruno desthuilliers

Reputation: 77952

First (slightly unrelated but anyway...) you have an error in your config: your root logger is defined at the top-level instead of being under the loggers key.

Then you may want to set the root logger's level way higher (WARNING or ERROR - WARNING being the default and a sensible one) so you only get the most important messages from other lib's loggers.

Once done with it, you can also use two distinct handlers - one for the root logger and a dedicated one for your own libs - so you get one log with important messages from all libs (including yours) and another log with all messages from your own libs and nothing from the others.

Upvotes: 1

Paolo Venturi
Paolo Venturi

Reputation: 1

You have to set disable_existing_loggers to true in your logging.json.

Upvotes: 0

Related Questions