log4net: how to customize data to be stored

Question

I have few different web services exposed by my web site. Here is a goals I want to reach:

1. Store all data in DB in one table;
2. Each record should have text field with 'service/component' name;
3. It is necessary to store all income parameters when any method is called;
4. It is necessary to store all outcome parameters AND error messages if any exception is occurred.

My original intention was to use log4net. But from these walk-through (http://sadi02.wordpress.com/2008/09/15/how-to-store-log-in-database-using-log4net/, http://logging.apache.org/log4net/release/config-examples.html) I don't see an easy way to add custom data into log table.

Is it feasible to use log4net for such purposes? How can I change logging table schema to store custom data? Should I modify source code of log4net library? Probably would be better to write a custom functionality to store such kind of data?

Thanks.

Answer 1

Neither of suggested answers works for me. The problem is that I need to have several custom fields. For me it seems too complicated to create different instances of logger or to setup column values each time through the 'properties'.

I will go (actually, already implemented) custom logger that put data in DB.

Answer 2

Looking at the log4net documentation, can you figure out how to get your custom data logged to a file? If so, you should also be able to get that same data logged to the database. Just add more columns in your table, more columns in the <command text> node of the AdoNetAppender, and more <parameter> nodes.

I think that you will have some work to do to get your incoming and outgoing parameters logged. Do you need them logged in separate columns (probably not easy to do cleanly)? Is it ok if they are logged together with the message?

For example, if you have the following method that you want put logging into:

public void DoSomething(int x, int y)
{
  log.Info("Inside DoSomething");
}

What do you want your output to look like? Do you want the "standard" log4net info to appear in separate columns (timestamp, loggername, level, message)? What about the parameters? Should x and y appear in separate columns (probably not very easy to do unless every method has the same number of parameters) or would it ok if the parameters were logged like this:

public void DoSomething(int x, int y)
{
  ILog log = LogManager.GetLogger("abc");
  log.InfoFormat("Parameters: x = {0}, y = {1}", x, y);
  log.Info("Inside DoSomething");
}

The log statements will generate messages that will look something like this:

11/29/2010 16:36:00 | abc | INFO | Parameters: x = 10, y = 20
11/29/2010 16:36:00 | abc | INFO | Inside DoSomething

I have used the | character to show what the fields would be with a pattern layout that shows the timestamp, loggername, log level, and message.

Looking at an AOP solution like PostSharp might help you because you could relatively easily add enter/exit logging without "polluting" your application source code with logging statements. Inside of the logging "aspect" you would have access to the parameters to the method.

I might be missing something, but I suspect that if you want to maintain your method parameters as individual columns in the database then you are going to find it hard to do. If you are satisfied with combining all parameters (manually) for each method as a single column (essentially a formatted string), then that will be easier. One price you will have to pay is that you will have to explicitly log all parameters (unless you go the AOP route).

If you are considering using log4net, you should also consider using NLog. It won't necessarily help you with the issues that I described above, but I think that it is a worthy competitor to log4net.

[EDIT]

To get the "component name" logged by log4net, you should name your loggers for your components. When you call LogManager.GetLogger(name), you can pass any name (or type). A common pattern is to have code like this in each class:

public class MyClass
{
  private static readonly ILog logger = LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);

  public void DoSomething(int x)
  {
    logger.InfoFormat("Inside DoSomething.  x = {0}", x);
  }
}

This will get a logger named with the fully qualified class name (namespace + class name). If you do this in every class, then you can control logging (level, which appender it goes to, etc) per class. So, you could easily turn logging for Class1 to Info and logging for Class2 Off, etc. This gives you the potential for the greatest degree of control of your logging.

Now, in your config file, you are not obligated to list each logger (i.e. each fully qualified class name) explicitly. You could just configure the root logger and then those settings would be applied to every logger. You could control you logging by namespace. For example, if you work for CompanyX and you have explicit namespace rules, your namespaces might look like:

CompanyX
CompanyX.DataAccess
CompanyX.DataAccess.Read
CompanyX.DataAccess.Write
CompanyX.GUI
CompanyX.Forms
CompanyX.Controls

Within each namespace, you might have various class (like several different data readers, helper classes, data writers, etc). With you classes organized like this and with your classes getting loggers as above, you could easily configure logging for all classes in CompanyX, or all loggers in CompanyX.DataAccess, or CompanyX.DataAccess.Read, etc. You could even turn all logging off, but turn it on for that one troublesome class that is giving you problems.

You can also get your loggers by arbitrary names (i.e. you do not have to use class name if you don't want to). You could define functional areas in your app(s) and get loggers based on that:

ILog logger = LogManager.GetLogger("DataAccess");
ILog logger = LogManager.GetLogger("Performance");
ILog logger = LogManager.GetLogger("UI");

And so on. I don't see that this gives much benefit over using the fully qualified class name. If your namespace(s) are well organized, then your ability to configure your logging will have maximum flexibility with minimum effort on your part.

Another word on NLog... NLog is very similar to log4net. It does have the useful feature of being able to automatically return a logger for the current class:

Logger logger = NLog.LogManager.GetCurrentClassLogger();

This at least saves some typing on your part. NLog has also just come out with a new release (currently in beta).

Don't know if any of this helped, but I hope it did!

Good luck!

Answer 3

According to this discussion, the process is pretty straightforward.

The first step is to update the command text in the appender declaration into your config file:

<commandText value="INSERT INTO Log4Net ([Date],[Thread],[Level],[Logger],[Message],[Exception],[MyColumn]) VALUES (@log_date, @thread, @log_level, @logger, @message, @exception,@MyColumn)"/>        

The next step is to and add a new parameter definition for the custom column:

<parameter>
   <parameterName value="@MyColumn "/>
   <dbType value="String" />
   <size value="255" />
   <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%property{MyColumn}" />
   </layout>
</parameter>

Finally, you can access the value through the GlobalContext properties of log4net:

log4net.GlobalContext.Properties["MyColumn"] = "MyValue";
log.Debug("My message");

You can do this for as many fields as you need.