CODEWITHSUNDEEP
python
Hary
Hary

Reputation: 1218

How to add custom data in Python logs

I am using python logging.

Once I get a handle to logger object, I do the following to log

logger.info("this is my log message")

My requirement is this:

  1. In the above custom log info message, I would want to pass more parameters like the following: 

    custom_name = "DISPLAY"
logger.info("this is my log message", custom_name)

  2. custom_name parameter should be consumed by the log formatter's custom_name placeholder as shown below

    formatter = logging.Formatter('%(asctime)s %(custom_name)s:%(name)s %(message)s')

I read articles, stackoverflow posts, python logging documentation. I could only find how to create custom placeholders in the formatter. I am not sure how to pass those custom placeholder values from the log statement itself. Can someone point me to a resource or let me know how to accomplish this ?

Edit:

I could do something like this

import logging

CUSTOM_VAR= 'myCustomVarValue'


class ContextFilter(logging.Filter):

    def filter(self, record):
        record.CUSTOM_VAR= CUSTOM_VAR
        return True

FORMAT = '%(CUSTOM_VAR)s %(asctime)s - %(levelname)s - %(message)s'
logging.basicConfig(format=FORMAT, datefmt='%d/%m/%Y %H:%M:%S')

logger = logging.getLogger(__name__)
logger.addFilter(ContextFilter())

logger.info("'this is info message")

The above works fine. However, CUSTOM_VAR is a static value. How can I dynamically pass this value from log.info ??

Upvotes: 0

Views: 2398

Answers (2)

Kapil Ahuja
Kapil Ahuja

Reputation: 21

In config.py we can set CUSTOM_VAR using contextvars class and we can update dynamically inside the main code.

import contextvars
CUSTOM_VAR = contextvars.ContextVar("CUSTOM_VAR", default=None)

Inside main code we can update CUSTOM_VAR like this

CUSTOM_VAR.set("DISPLAY")

we can use set CUSTOM_VAR inside logger file

import logging
from config import CUSTOM_VAR


class ContextFilter(logging.Filter):

    def filter(self, record):
        if not hasattr(record, 'CUSTOM_VAR'):
            record.custom_name= CUSTOM_VAR.get()
            return True

FORMAT = '%(custom_name)s %(asctime)s - %(levelname)s - %(message)s'
logging.basicConfig(format=FORMAT, datefmt='%d/%m/%Y %H:%M:%S')

logger = logging.getLogger(__name__)
logger.addFilter(ContextFilter())

logger.info("'this is info message",extra=CUSTOM_VAR)

Upvotes: 2

Hary
Hary

Reputation: 1218

I was able to accomplish it the following way:

In log config code, have your formatter declared like this:

 formatter = logging.Formatter(
   '%(asctime)s %(levelname)s %(name)s,%(traceId)s %(packageName)s %(message)s')

Now in the code, where you have the log statements, have such statements written like this where you pass extra param values dynamically:

logger.info("custom log message goes here.....",
            extra={'traceId': 'XDDF4543GG', 'packageName': 'myPackage'})

corresponding log entry would look like this:

2017-05-15 17:59:24,101 INFO [my-proj-name,XDDF4543GG] myPackage custom log message goes here.....

Also, if you don't put all extra params in log.info then you would get a keyError error message.

Upvotes: 0

Related Questions