How to setup Serilog with Azure Functions v4 correctly?

Question

I want to use Serilog in an Azure Function v4 (.net 6) (the logs should be sent to Datadog). For this I have installed the following nuget packages:

<PackageReference Include="Serilog" Version="2.10.0" />
<PackageReference Include="Serilog.Extensions.Logging" Version="3.1.0" />
<PackageReference Include="Serilog.Formatting.Compact" Version="1.1.0" />
<PackageReference Include="Serilog.Sinks.Console" Version="4.0.1" />
<PackageReference Include="Serilog.Sinks.Datadog.Logs" Version="0.3.5" />

Below is the configuration in the Startup.cs class:

public override void Configure(IFunctionsHostBuilder builder)
{
  builder.Services.AddHttpClient();
  
  //... adding services etc.

  Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
    .MinimumLevel.Override("Worker", LogEventLevel.Warning)
    .MinimumLevel.Override("Host", LogEventLevel.Warning)
    .MinimumLevel.Override("System", LogEventLevel.Error)
    .MinimumLevel.Override("Function", LogEventLevel.Error)
    .MinimumLevel.Override("Azure.Storage.Blobs", LogEventLevel.Error)
    .MinimumLevel.Override("Azure.Core", LogEventLevel.Error)
    .Enrich.WithProperty("Application", "Comatic.KrediScan.AzureFunctions")
    .Enrich.FromLogContext()
    .WriteTo.DatadogLogs("XXXXXXXXXXX", configuration: new DatadogConfiguration() { Url = "https://http-intake.logs.datadoghq.eu" }, logLevel:   LogEventLevel.Debug)
    .WriteTo.Console()
    .CreateLogger();

  builder.Services.AddSingleton<ILoggerProvider>(sp => new SerilogLoggerProvider(Log.Logger, true));

  builder.Services.AddLogging(lb =>
  {
    //lb.ClearProviders(); //--> if used nothing works...
    lb.AddSerilog(Log.Logger, true);
  });

Basically logging works, but all log statements are written twice (with a few milliseconds difference, Datadog and Console).

Obviously I am doing something fundamentally wrong with the configuration. I don't use appsettings.json, the configuration of Serilog takes place exclusively in the code. I have scoured the entire internet and read just about every article on Serilog and Azure Functions. On Stackoverflow I also read virtually every question about it and tried all the answers. Unfortunately, so far without success.

SO-Questions for example: Use Serilog with Azure Log Stream
How do I use Serilog with Azure WebJobs?
Serilog enricher Dependency Injection with Azure Functions
https://github.com/hgmauri/sample-azure-functions/blob/main/src/Sample.AzureFunctions.DotNet31/Startup.cs

Is there any example for setting up Serilog with Azure Functions v4 / .net 6?

Thanks a lot for the help!
Michael Hachen

Answer 1

I'm answering it for Azure Functions v4 (Isolated mode).

Firstly, the Functions.csproj file structure is outlined below, allowing one to add the Serilog packages I am using.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <LangVersion>latest</LangVersion>
    <AzureFunctionsVersion>v4</AzureFunctionsVersion>
    <OutputType>Exe</OutputType>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>
  <ItemGroup>
    <FrameworkReference Include="Microsoft.AspNetCore.App" />
    <!-- Application Insights isn't enabled by default. See https://aka.ms/AAt8mw4. -->
    <!-- <PackageReference Include="Microsoft.ApplicationInsights.WorkerService" Version="2.22.0" /> -->
    <!-- <PackageReference Include="Microsoft.Azure.Functions.Worker.ApplicationInsights" Version="2.0.0" /> -->
    <PackageReference Include="Microsoft.Azure.Functions.Worker" Version="2.0.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore" Version="2.0.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.ServiceBus" Version="5.22.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="2.0.0" />
    <PackageReference Include="Polly" Version="8.5.1" />
    <PackageReference Include="Refit.HttpClientFactory" Version="8.0.0" />
    <PackageReference Include="Serilog" Version="4.2.0" />
    <PackageReference Include="Serilog.Extensions.Logging" Version="9.0.0" />
    <PackageReference Include="Serilog.Settings.Configuration" Version="9.0.0" />
    <PackageReference Include="Serilog.Sinks.Console" Version="6.0.0" />
    <PackageReference Include="Serilog.Sinks.Seq" Version="9.0.0" />
  </ItemGroup>
  <ItemGroup>
    <None Update="host.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="local.settings.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <CopyToPublishDirectory>Never</CopyToPublishDirectory>
    </None>
  </ItemGroup>
  <ItemGroup>
    <Using Include="System.Threading.ExecutionContext" Alias="ExecutionContext" />
  </ItemGroup>
</Project>

Second Program.cs as below

using ADS.Functions.Http;
using Microsoft.Azure.Functions.Worker.Builder;
using Microsoft.Extensions.Hosting;
using Serilog;

var builder = FunctionsApplication.CreateBuilder(args);
builder.Services.ConfigureServices(builder.Configuration);

try
{
    var app = builder.Build();
    Log.Information("Starting ADS.Functions...");
    app.Run();
}
catch (Exception ex)
{
    Log.Fatal(ex, "ADS.Functions startup failed.");
}
finally
{
    Log.CloseAndFlush();
}

And then service extensions are shown below.

using System.Text.Json;
using ADS.Functions.Services;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Refit;
using Serilog;

namespace ADS.Functions.Http;

public static class ServiceExtensions
{
    public static void ConfigureServices(this IServiceCollection services, IConfiguration configuration)
    {
        services.ConfigureLogging(configuration);        
    }

    private static void ConfigureLogging(this IServiceCollection services, IConfiguration configuration)
    {
        Log.Logger = new LoggerConfiguration()
            .ReadFrom.Configuration(configuration.GetSection("Serilog"))
            .Enrich.FromLogContext()
            .Enrich.WithProperty("ApplicationName", "ADS Functions")
            .WriteTo.Console()
            .CreateLogger();

        services.AddLogging(loggingBuilder =>
        {
            loggingBuilder.ClearProviders(); // Removes default Azure Functions logging
            loggingBuilder.AddSerilog();
        });
    }

    private static void ConfigureMtpApiHttpClient(this IServiceCollection services, IConfiguration configuration)
    {
        var apiBaseUrl = GetMtpApiBaseUrl(configuration);
        var refitSettings = CreateRefitSettings();

        services.AddRefitClient<IMtpApiAuthentication>(refitSettings)
            .ConfigureHttpClient(client => client.BaseAddress = apiBaseUrl);

        services.AddTransient<MtpAuthenticationHandler>();

        services.AddRefitClient<IMtpApi>(refitSettings)
            .ConfigureHttpClient(client => client.BaseAddress = apiBaseUrl)
            .AddHttpMessageHandler<MtpAuthenticationHandler>();
    }

    private static Uri GetMtpApiBaseUrl(IConfiguration configuration)
    {
        var baseUrl = configuration["MtpApi:BaseUrl"];
        if (string.IsNullOrWhiteSpace(baseUrl))
        {
            Log.Fatal("MTP API Base URL is missing in configuration.");
            throw new InvalidOperationException("MTP API Base URL is required.");
        }

        return new Uri(baseUrl);
    }

    private static RefitSettings CreateRefitSettings()
    {
        var jsonOptions = new JsonSerializerOptions
        {
            PropertyNamingPolicy = null // Ensures PascalCase serialization
        };

        return new RefitSettings
        {
            ContentSerializer = new SystemTextJsonContentSerializer(jsonOptions)
        };
    }
}

Finally the local.settings.json file as below

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated"    
  },
  "Serilog": {
    "Using": ["Serilog.Sinks.Console", "Serilog.Sinks.Seq"],
    "LevelSwitches": {
      "$appLogLevel": "Information",
      "$seqSwitch": "Information",
      "$consoleSwitch": "Information"
    },
    "MinimumLevel": {
      "ControlledBy": "$appLogLevel",
      "Override": {
        "Default": "Warning",
        "Host": "Warning",
        "Function": "Warning",
        "System": "Warning",
        "Microsoft": "Warning",
        "Azure.Core": "Warning",
        "Worker": "Warning",
        "Microsoft.Hosting.Lifetime": "Information",
        "Microsoft.EntityFrameworkCore": "Warning",
        "Microsoft.AspNetCore.Authentication": "Warning"
      }
    },
    "WriteTo": [
      {
        "Name": "Console",
        "Args": {
          "levelSwitch": "$consoleSwitch",
          "theme": "Serilog.Sinks.SystemConsole.Themes.SystemConsoleTheme::Literate, Serilog.Sinks.Console",
          "outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {SourceContext} {Message:lj}{NewLine}{Exception}"
        }
      },
      {
        "Name": "Seq",
        "Args": {
          "levelSwitch": "$seqSwitch",
          "serverUrl": "http://localhost:5341",
          "outputTemplate": "{Timestamp:HH:mm:ss} [{Level:u3}] {SourceContext} {Message}{NewLine}{Exception} {Properties:j}",
          "shared": true
        }
      }
    ],
    "Enrich": ["FromLogContext", "WithMachineName"]
  }
}

If configured correctly, you will only see a single log from Serilog.

Answer 2

For a Functions v4/ Isolated .NET 7 version of this

builder.ConfigureServices((hostingContext, services) =>
{ 
    Log.Logger = new LoggerConfiguration()
        .ReadFrom.Configuration(config)
        .Enrich.FromLogContext()
        .WriteTo.Console()
        .WriteTo.ApplicationInsights(TelemetryConfiguration.CreateDefault(), TelemetryConverter.Traces)
        .CreateLogger();

    services.AddLogging(lb => lb.AddSerilog(Log.Logger, true));
    services.AddApplicationInsightsTelemetryWorkerService();
    services.AddInterventionCalculatorServices(config);//injecting my services
});

Gets you serilog with sinks for console and Insights.

You can also configure the default logger within the hosts.json file

{
  "version": "2.0",
  "logging": {
    "logLevel": {
      "Function.MyFunctionName.User": "Information",
      "Function": "Error"
    }
}

To remove duplicate messages (there may be a way to turn off console entirely from here, but I've not found one).

After this, you should only be seeing your serilog sinks and only 1 set of the important console messages.

Note that recent versions of Visual studio have had some issues with copying files on "F5" (latest patch 17.7.0 Preview 3.0 to VS Community is supposed to resolve this, but I'm not sure its 100%). Make sure to rebuild after changing your hosts.json file and to verify whats in your deployed folder to retain your sanity...

Answer 3

To set up Serilog.ILogger instead of ILogger<T> you can add Serilog as Singleton

builder.AddSingleton<ILogger>(Log.Logger);

instead of

services.AddLogging(configure => configure.AddSerilog(Log.Logger));

and then in the function inject ILogger from Serilog instead of ILoggerFactory and log as

_logger.Information("awesome log");

I run isolated function in .net7

Answer 4

Got it! After replacing all ILogger with ILogger<T> and removing the line builder.Services.AddSingleton<ILoggerProvider>(sp => new SerilogLoggerProvider(Log.Logger, true)); everything worked as expected.

Startup.cs

Log.Logger = new LoggerConfiguration()
          .MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
          .MinimumLevel.Override("Worker", LogEventLevel.Warning)
          .MinimumLevel.Override("Host", LogEventLevel.Warning)
          .MinimumLevel.Override("System", LogEventLevel.Error)
          .MinimumLevel.Override("Function", LogEventLevel.Error)
          .MinimumLevel.Override("Azure.Storage.Blobs", LogEventLevel.Error)
          .MinimumLevel.Override("Azure.Core", LogEventLevel.Error)
          .Enrich.WithProperty("Application", $"xxxxx.AzureFunctions.{builder.GetContext().EnvironmentName}")
          .Enrich.FromLogContext()
          .Enrich.WithExceptionDetails(new DestructuringOptionsBuilder()
            .WithDefaultDestructurers()
            .WithDestructurers(new[] { new SqlExceptionDestructurer() }))
          .WriteTo.Seq(builder.GetContext().EnvironmentName.Equals("Development", StringComparison.OrdinalIgnoreCase) ? "http://localhost:5341/" : "https://xxxxxx.xx:5341/", LogEventLevel.Verbose)
          .WriteTo.Console(theme: SystemConsoleTheme.Literate)
          .CreateLogger();
      
      builder.Services.AddLogging(lb =>
      {
        lb.AddSerilog(Log.Logger, true);
      });

Answer 5

For those that got here because they can't properly config their logging in an azure function with the app insights sink this is what works for me:

private static void ConfigureLogging(IServiceCollection services)
{
   Log.Logger = new LoggerConfiguration()
        .MinimumLevel.Information()
        .MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
        .MinimumLevel.Override("System", LogEventLevel.Warning)
        .MinimumLevel.Override("Worker", LogEventLevel.Warning)
        .MinimumLevel.Override("Host", LogEventLevel.Warning)
        .MinimumLevel.Override("Function", LogEventLevel.Warning)
        .MinimumLevel.Override("Azure", LogEventLevel.Warning)
        .MinimumLevel.Override("DurableTask", LogEventLevel.Warning)
        .Enrich.FromLogContext()
        .Enrich.WithExceptionDetails()
        .WriteTo.ApplicationInsights(
            TelemetryConfiguration.CreateDefault(),
            TelemetryConverter.Events,
            LogEventLevel.Information)
        .CreateLogger();
        services.AddLogging(configure => configure.AddSerilog(Log.Logger));
}

The example here at the time of writing doesn't seem to work. The logging scope doesn't get captured in the output.

Serilog Version: 2.11.0

Serilog.Sinks.ApplicationInsights Version: 4.0.0

Linked example for the future:

[assembly: FunctionsStartup(typeof(MyFunctions.Startup))]
namespace MyFunctions
{
    public class Startup : FunctionsStartup
    {
        public override void Configure(IFunctionsHostBuilder builder)
        {
            builder.Services.AddSingleton<ILoggerProvider>((sp) => 
            {
                Log.Logger = new LoggerConfiguration()
                    .Enrich.FromLogContext()
                    .WriteTo.ApplicationInsights(sp.GetRequiredService<TelemetryClient>(), TelemetryConverter.Traces)
                    .CreateLogger();
                return new SerilogLoggerProvider(Log.Logger, true);
            });
        }
    }
}