CODEWITHSUNDEEP
.netperformancelogginglog4net
geva30
geva30

Reputation: 345

high speed logging using log4net

I am interested in very high speed logging within log4net(around 10K messages per second). to that end i thought of implementing the following modules:

  1. protocol buffers based Layout (IRawlayout) - for superior serialization performance
  2. shared memory appender and Plugin - to reduce IPC between the logging application and the logging server.

is this the way to integrate those technologies?

i also considered using ETW within log4net but they seem to be too different to be elegantly intergrated.

Upvotes: 1

Views: 1267

Answers (2)

chronoxor
chronoxor

Reputation: 3559

Some time ago I made investigation of log4net performance and created a blog post

You can find there several log4net async forwarding appenders:

  1. Asynchronous log4net solution 1: BlockingCollection
  2. Asynchronous log4net solution 2: Hot swap technique
  3. Asynchronous log4net solution 3: .NET port of LMAX Disruptor

Performance results of asynchronous log4net solutions with RollingFileAppender:

enter image description here enter image description here enter image description here

And my some notes about log4net performance:

  • AsyncForwardingAppenderBlocking solution based on BlockingCollection performs very bad, especially for several logging producers.
  • AsyncForwardingAppenderHotSwap solution based on Hot swap technique performs the best! However it might requires additional memory and even causes OutOfMemoryException when logging events are produced very fast!
  • AsyncForwardingAppenderDisruptor solution based on .NET port of LMAX Disruptor performs good, but it is still far from log4j2 even with NullAppender. It is a good point to investigate possible performance issues in disruptor-net or its wrong usage for the logging case.

Upvotes: 2

Dima
Dima

Reputation: 4128

I once looked at google protobuffer and came to a conclusion that it won't be as great help with logging as it seems at first. Logging involves lots of text, which is same text everywhere anyway. So, portability of protobuffers is not an advantage. As to the speed I'm also not sure, you still have to transmit the same text over the wire to the server either packed into a protobuffer packet or tagged by xml. This, of course, is relevant if you are logging textual information. In case of binary logging it would probably be a cool thing to do though.

Upvotes: 1

Related Questions