log json string message using log4j in JSON format without breaking the format with quotes

Question

I am trying to format logs in a json format and using pattern layout to do so. But the output getting generated is not in a proper json format because of the extra quotes within a msg is getting logged.

Below is the log4j.properties file =======================log4j.properties==================

appender.file.layout.pattern={"level":"%p","time":"%d{ISO8601}","thread":"%t","file":"%F", "line":"%L","message":"%m"}%n
loggers=file
logger.file.name=com.ibm
logger.file.level = error
logger.file.level = debug
logger.file.appenderRefs = file
logger.file.appenderRef.file.ref = LOGFILE
rootLogger.level = error
rootLogger.level = debug
rootLogger.appenderRefs = stdout
rootLogger.appenderRef.stdout.ref = STDOUT

Below is the coding format

String msg = "!@#$%^&*()_+=~{}[]|<>?;'Text with special character /\"\'\b\f\t\r\n";
// to escape especial character, serializing msg with jackson library method
Serialize serializeMsg = objectMapper.writeValueAsString(msg);
String jsonString = "" + serializeMsg + "\", \"crn\":\"" + crn + "\", \"otherProperty\":\"" + crn + "\"";
log.debug(serializeMsg );

Json log generated

{"level":"DEBUG","time":"2021-12-20T20:05:14,465","thread":"main","file":"App.java", "line":"81","message":""!@#$%^&*()_+=~{}[]|<>?;'Text with special character /\"'\b\f\t\r\n"", "crn":"5741af48-808a-4e55-8f37-681f025b1ce3", "otherProperty":"5741af48-808a-4e55-8f37-681f025b1ce3""}

How can I remove extra quotes from the message? Thanks!!

Answer 1

To answer your question directly: objectMapper.writeValueAsString(msg) outputs a valid JSON, therefore it is a string enclosed in double quotes. The message you constructed in your code is:

"!@#$%^&*()_+=~{}[]|<>?;'Text with special character /\"'\b\f\t\r\n"", "crn":"5741af48-808a-4e55-8f37-681f025b1ce3", "otherProperty":"5741af48-808a-4e55-8f37-681f025b1ce3"

and your PatternLayout encloses it in another pair of double quotes ("%m"). You can easily remove some double quotes from your code to make it a valid JSON message.

However the whole approach you took to solve your main problem (formatting log messages as JSON) is unstable (it's a big hack): you require all log messages in your application to have a JSON-resembling format. What happens if you forget to format a message correctly? What about library messages that certainly don't follow your format? What happens if your application's user want to change the logging layout? Your approach might also be vulnerable to log injection attacks if you are not careful about what you are logging.

A standard approach would be to let Log4j format your messages as JSON. There is a JSON Template Layout for that. You just need to:

1. Add the log4j-layout-template-json to your dependencies,

2. Create in your applications classpath a template (let's call it logTemplate.json):

   {
       "level": {
           "$resolver": "level",
           "field": "name"
       },
       "time": {
           "$resolver": "timestamp"
       },
       "thread": {
           "$resolver": "thread",
           "field": "name"
       },
       "file": {
           "$resolver": "source",
           "field": "fileName"
       },
       "line": {
           "$resolver": "source",
           "field": "lineNumber"
       },
       "message": {
           "$resolver": "message",
           "stringified": true
       }
   }
   

3. Update your log4j2.properties file to use the JSON Template Layout:

   appender.file.layout.type=JsonTemplateLayout
   appender.file.layout.eventTemplateUri=classpath:logTemplate.json
   appender.file.layout.locationInfoEnabled=true
   

4. Log messages by simply calling:

   log.debug("!@#$%^&*()_+=~{}[]|<>?;'Text with special character /\"\'\b\f\t\r\n");