CODEWITHSUNDEEP
javalog4j
CarlG
CarlG

Reputation: 1666

Log4j has no support for binary logging format?

I have a need to have Log4j log into a binary format so the logs can be easily machine-interpreted. I know I could use XML for this purpose but don't want the associated file size bloat or parsing overhead.

The whole Layout system seems to be inherently String based, which means I can't use the nice file-rolling appenders. LoggingEvent is serializable, so I'm thinking of writing a binary file appender myself, but I feel like I shouldn't be treading new ground here.

Am I missing something obvious?

Upvotes: 7

Views: 2567

Answers (5)

Sasha O
Sasha O

Reputation: 3749

I had similar problem myself and resolved it by using JSON format which is while texttual still arguably faster and easier to process than XML.

In log4j.properties, I set the following format:

log4j.appender.A1.layout.ConversionPattern = {"time":"%d", "msg": %m},%n

which makes it valid JSON, while the message itself is a JSON object as well, like:

myLogger.info("{...the contents of the object...}")

In my case the contents of the message is created from Java object using Jackson serializer, like so:

static ObjectMapper MAPPER= new ObjectMapper();
MAPPER.writeValueAsString(myObject);

The resulting log files can be processed with any problem that understand JSON, it just needs to be wrapped into

 [ <<file contents>>, {}]

Upvotes: 2

DwB
DwB

Reputation: 38320

Oddly enough, text is machine readable. Consider just setting you logging format to something that is easy to parse and you will end up with a log that is both human and machine readable.

Upvotes: 0

Asaf
Asaf

Reputation: 6510

I had a very similar problem recently. I had to write the log line to binary format but also wanted to have all the other appenders work normally.

What I did was implement my own appender which extends the AppenderSkeleton and takes the LoggingEvent object where the message returned by getMessage() is my own object.

I also write a class implementing ObjectRenderer which a Layout class of the appender will call in order to transform the logging object (the one I logged) to string.

Then for my own appender (which doesn't have a layout) the message is serialized into binary form and written to some byte stream. For other appenders the layout object calls my object renderer and the message is serialized to String.

In this way all common appenders work the same and I'm still able to append to my own format.

In Summary:

  • Write own logging object used this way: logger.info(LogEntry)
  • Implement ObjectRenderer to transform LogEntry to String
  • Extend AppenderSkeleton with my own BinaryFormatAppender

I don't know of any ready made solution for this.

Upvotes: 6

Mikael Vandmo
Mikael Vandmo

Reputation: 945

There is nothing in the log4j distribution.

You could have a look at http://sourceforge.net/projects/bclf/ but you will probably end up writing your own Appender and implement doAppend(LogginEvent).

Upvotes: 1

Andrew White
Andrew White

Reputation: 53516

As long as the logs are "regular" then a machine can parse a regex rather easily. If you are worried about size and XML you can zip the logs.

Upvotes: 0

Related Questions