Serilog - With AppSettings Config how do I configure Sub Loggers & Include Certain Namespaces only?

Question

I am trying to setup Serilog in a CMS that has some default logging configuration setup that we define as the CMS, however allowing developers using the CMS to extend and configure there own logging requirements by using the Serilog AppSettings Nuget package - https://github.com/serilog/serilog-settings-appsettings

I have some of this working and able to configure other Sinks in an external configuration file the problem I have and need help with, is how do I let developers configure a file sink to generate a txt logile that only includes their namespace?

With a C# class I know I can create a sub-logger and then use a Filter like so .Filter.ByIncludingOnly(Matching.FromSource("DevelopersNamespace")) but using the Serilog Analyzer VS Extension - https://github.com/Suchiman/SerilogAnalyzer it cannot generate an example XML AppSettings configuration.

Here is a copy of my Logger Configuration in C#

Serilog.Debugging.SelfLog.Enable(msg => System.Diagnostics.Debug.WriteLine(msg));

//Set this environment variable - so that it can be used in external config file
//add key="serilog:write-to:RollingFile.pathFormat" value="%BASEDIR%\logs\log-{Date}.txt" />
Environment.SetEnvironmentVariable("BASEDIR", AppDomain.CurrentDomain.BaseDirectory, EnvironmentVariableTarget.Process);

Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Debug() //Set to highest level of logging (as any sinks may want to restrict it to Errors only)
    .Enrich.WithProcessId()
    .Enrich.WithProcessName()
    .Enrich.WithThreadId()
    .Enrich.WithProperty("AppDomainId", AppDomain.CurrentDomain.Id)
    .Enrich.WithProperty("AppDomainAppId", HttpRuntime.AppDomainAppId.ReplaceNonAlphanumericChars(string.Empty))
    .Enrich.With<Log4NetLevelMapperEnricher>()

    //Main .txt logfile - in similar format to older Log4Net output
    //Ends with ..txt as Date is inserted before file extension substring
    .WriteTo.File($@"{AppDomain.CurrentDomain.BaseDirectory}\App_Data\Logs\UmbracoTraceLog.{Environment.MachineName}..txt",
        rollingInterval: RollingInterval.Day,
        restrictedToMinimumLevel: LogEventLevel.Debug,
        retainedFileCountLimit: null, //Setting to null means we keep all files - default is 31 days
        outputTemplate: "{Timestamp:yyyy-MM-dd HH:mm:ss,fff} [P{ProcessId}/D{AppDomainId}/T{ThreadId}] {Log4NetLevel}  {SourceContext} - {Message:lj}{NewLine}{Exception}")

    //.clef format (Compact log event format, that can be imported into local SEQ & will make searching/filtering logs easier)
    //Ends with ..txt as Date is inserted before file extension substring
    .WriteTo.File(new CompactJsonFormatter(), $@"{AppDomain.CurrentDomain.BaseDirectory}\App_Data\Logs\UmbracoTraceLog.{Environment.MachineName}..json", 
        rollingInterval: RollingInterval.Day, //Create a new JSON file every day
        retainedFileCountLimit: null, //Setting to null means we keep all files - default is 31 days
        restrictedToMinimumLevel: LogEventLevel.Debug)

    //Read any custom user configuration of logging from serilog config file
    .ReadFrom.AppSettings(filePath: AppDomain.CurrentDomain.BaseDirectory + @"\config\serilog.config")
    .CreateLogger();

Here is an example of the AppSettings configuration file that users will be able to modify their own sinks with.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <appSettings>
        <!-- Controls log levels for all sinks (Set this higher than child sinks) -->
        <add key="serilog:minimum-level" value="Verbose" />

        <!-- Write to a user log file -->
        <add key="serilog:using:File" value="Serilog.Sinks.File" />
        <add key="serilog:write-to:File.path" value="%BASEDIR%\logs\warren-log.txt" /><!-- Can we do a relative path to website ? -->
        <add key="serilog:write-to:File.restrictedToMinimumLevel" value="Debug" /> 
        <add key="serilog:write-to:File.retainedFileCountLimit" value="32" /> <!-- Number of log files to keep (or remove value to keep all files) -->
        <add key="serilog:write-to:File.rollingInterval" value="Day" /> <!-- Create a new log file every Minute/Hour/Day/Month/Year/infinite -->

        <!-- TODO: How do I filter the file sink for customer to their own namespace ?? -->

    </appSettings>
</configuration>

I am open to ideas and suggestions on how I can achieve this with a goal of allowing developers to configure their own sinks and to optionally filter to their own namespace if they wish to (as I doubt users will want to write their own sink code)

Answer 1

For anyone interested or was to come across this post at a later date this is how I solved it.

I used two configuration files one to configure the main logging pipeline and a user config for a sub-logger that they can then use filtering if required without effecting the main logging pipeline.

Serilog.Debugging.SelfLog.Enable(msg => System.Diagnostics.Debug.WriteLine(msg));

//Set this environment variable - so that it can be used in external config file
//add key="serilog:write-to:RollingFile.pathFormat" value="%BASEDIR%\logs\log.txt" />
Environment.SetEnvironmentVariable("BASEDIR", AppDomain.CurrentDomain.BaseDirectory, EnvironmentVariableTarget.Process);

Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Verbose() //Set to highest level of logging (as any sinks may want to restrict it to Errors only)
    .Enrich.WithProcessId()
    .Enrich.WithProcessName()
    .Enrich.WithThreadId()
    .Enrich.WithProperty("AppDomainId", AppDomain.CurrentDomain.Id)
    .Enrich.WithProperty("AppDomainAppId", HttpRuntime.AppDomainAppId.ReplaceNonAlphanumericChars(string.Empty))
    .Enrich.With<Log4NetLevelMapperEnricher>()

    //Main .txt logfile - in similar format to older Log4Net output
    //Ends with ..txt as Date is inserted before file extension substring
    .WriteTo.File($@"{AppDomain.CurrentDomain.BaseDirectory}\App_Data\Logs\UmbracoTraceLog.{Environment.MachineName}..txt",
        rollingInterval: RollingInterval.Day,
        restrictedToMinimumLevel: LogEventLevel.Verbose,
        retainedFileCountLimit: null, //Setting to null means we keep all files - default is 31 days
        outputTemplate: "{Timestamp:yyyy-MM-dd HH:mm:ss,fff} [P{ProcessId}/D{AppDomainId}/T{ThreadId}] {Log4NetLevel}  {SourceContext} - {Message:lj}{NewLine}{Exception}")

    //.clef format (Compact log event format, that can be imported into local SEQ & will make searching/filtering logs easier)
    //Ends with ..txt as Date is inserted before file extension substring
    .WriteTo.File(new CompactJsonFormatter(), $@"{AppDomain.CurrentDomain.BaseDirectory}\App_Data\Logs\UmbracoTraceLog.{Environment.MachineName}..json", 
        rollingInterval: RollingInterval.Day, //Create a new JSON file every day
        retainedFileCountLimit: null, //Setting to null means we keep all files - default is 31 days
        restrictedToMinimumLevel: LogEventLevel.Verbose)

    //Read from main serilog.config file
    .ReadFrom.AppSettings(filePath: AppDomain.CurrentDomain.BaseDirectory + @"\config\serilog.config")

    //A nested logger - where any user configured sinks via config can not effect the main 'umbraco' logger above
    .WriteTo.Logger(cfg =>
        cfg.ReadFrom.AppSettings(filePath: AppDomain.CurrentDomain.BaseDirectory + @"\config\serilog.user.config"))
    .CreateLogger();

Here is then a sample of the two configuration files:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <appSettings>

        <!-- Used to toggle the loge levels for the main Umbraco log files -->
        <!-- Found at /app_data/logs/ -->
        <!-- NOTE: Changing this will also flow down into serilog.user.config -->
        <add key="serilog:minimum-level" value="Verbose" />

        <!-- To write to new log locations (aka Sinks) such as your own .txt files, ELMAH.io, Elastic, SEQ -->
        <!-- Please use the serilog.user.config file to configure your own logging needs -->

    </appSettings>
</configuration>

And here is the configuration file where the user can then filter with their own namespace:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <appSettings>

        <!-- Controls log levels for all user-definied child sub-logger sinks configured here (Set this higher than child sinks) -->
        <add key="serilog:minimum-level" value="Verbose" />

        <!-- For Different Namespaces - Set different logging levels -->
        <add key="serilog:minimum-level:override:Microsoft" value="Warning" />
        <add key="serilog:minimum-level:override:Microsoft.AspNetCore.Mvc" value="Error" />
        <add key="serilog:minimum-level:override:YourNameSpace" value="Information" />

        <!-- All logs definied via user.config will contain this property (won't be in main Umbraco logs) -->
        <add key="serilog:enrich:with-property:websiteName" value="Warrens Website" />

        <!-- Write to a user log file -->
        <add key="serilog:using:File" value="Serilog.Sinks.File" />
        <add key="serilog:write-to:File.path" value="%BASEDIR%\logs\warren-log.txt" />
        <add key="serilog:write-to:File.restrictedToMinimumLevel" value="Debug" /> <!-- I will be ignored as Debug as the user logging pipleine has it min set to Information, so only Info will flow through me -->
        <add key="serilog:write-to:File.retainedFileCountLimit" value="32" /> <!-- Number of log files to keep (or remove value to keep all files) -->
        <add key="serilog:write-to:File.rollingInterval" value="Day" /> <!-- Create a new log file every Minute/Hour/Day/Month/Year/infinite -->

        <!-- Filters all above sink's to use this expression -->
        <!-- Common use case is to include SourceType starting with your own namespace -->
        <add key="serilog:using:FilterExpressions" value="Serilog.Filters.Expressions" />
        <add key="serilog:filter:ByIncluding.expression" value="StartsWith(SourceContext, 'MyNamespace')" />

    </appSettings>
</configuration>