Yehia A.Salam
Yehia A.Salam

Reputation: 2028

Excluding AppRequests in Serilog Azure App Insights Sink

I'm working on an ASP.NET application and using Azure AppInsights and Serilog Sink https://github.com/serilog-contrib/serilog-sinks-applicationinsights for pushing the logs. However one problem I have is I'm getting huge amounts of AppRequests (GET/POST) logged on AppInsights that I'm really not interested in, and costing me a lot of money on the Azure Subscription.

I tried to exclude the logs by Type field and checking for an attribute available only for the AppRequests table (Url), but both failed an I'm still getting all the AppRequests in App Insights, not sure what I'm doing wrong:

appsettings.json:

  "Serilog": {
      "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.ApplicationInsights", "Serilog.Expressions" ],
      "MinimumLevel": {
        "Default": "Debug",
        "Override": {
          "Microsoft.AspNetCore.Mvc.ModelBinding": "Warning",
          "Microsoft.AspNetCore.Routing": "Warning",
          "Microsoft.AspNetCore.Cors.Infrastructure.CorsService": "Warning",
          "Microsoft.EntityFrameworkCore.Infrastructure": "Warning"            
        }
      },
      "Filter":[
        {
          "Name": "ByExcluding",
          "Args": {
            "expression": "IsDefined(Url)" # I tried "Type = 'AppRequests'" but didn't work as well
          }
        }
      ],
      "WriteTo": [
        {
          "Name": "Console"
        },
        {
          "Name": "ApplicationInsights",
          "Args": {
            "connectionString" : "InstrumentationKey=7aa4cc33-fb03-4b4c-81b8-7a257c083a6e;IngestionEndpoint=https://westeurope-1.in.applicationinsights.azure.com/;LiveEndpoint=https://westeurope.livediagnostics.monitor.azure.com/",
            "restrictedToMinimumLevel": "Debug",
            "telemetryConverter": "Serilog.Sinks.ApplicationInsights.TelemetryConverters.TraceTelemetryConverter, Serilog.Sinks.ApplicationInsights"
          }
        }
      ],
      "Enrich": [ "FromLogContext" ]
    },

Screenshots from the AppRequests logs: enter image description here enter image description here

Upvotes: 0

Views: 50

Answers (2)

Chinmay T
Chinmay T

Reputation: 1133

You were very close. You might want to use the expression per below sample from Serilog.

Sample AppSettings.json config from GitHub Repo

{
  "Serilog": {
    "Using": ["Serilog.Expressions"],
    "Filter": [
      {
        "Name": "ByExcluding",
        "Args": {
          "expression": "RequestPath like '/health%'"
        }
      }
    ]
  }
}

Working example of Minimal API with net9.0 as TargetFramework.

NuGet packages installed

<PackageReference Include="Serilog" Version="4.2.0" />
<PackageReference Include="Serilog.AspNetCore" Version="9.0.0" />
<PackageReference Include="Serilog.Expressions" Version="5.0.0" />
<PackageReference Include="Serilog.Enrichers.Process" Version="3.0.0" />
<PackageReference Include="Serilog.Enrichers.Thread" Version="4.0.0" />
<PackageReference Include="Serilog.Formatting.Compact" Version="3.0.0" />
<PackageReference Include="Serilog.Settings.Configuration" Version="9.0.0" />
<PackageReference Include="Serilog.Sinks.Console" Version="6.0.0" />
<PackageReference Include="Serilog.Sinks.ApplicationInsights" Version="4.0.0" />
<PackageReference Include="Serilog.Sinks.File" Version="6.0.0" />

Program.cs

using Serilog;

var builder = WebApplication.CreateBuilder(args);

//removing the default logger
builder.Logging.ClearProviders();

//Configure Serilog
builder.Host.UseSerilog((hostingContext, loggerConfiguration) =>
{
    loggerConfiguration
        .ReadFrom.Configuration(hostingContext.Configuration)
        .Enrich.FromLogContext()
        .Enrich.WithProperty("Application", "MyApp.Api")
        .Enrich.WithProperty("Environment", hostingContext.HostingEnvironment.EnvironmentName)
        .Enrich.WithProperty("Version", "1.0.0")
        .Enrich.WithProperty("MachineName", Environment.MachineName);
        
});




builder.AddServiceDefaults();

// Add services to the container.
// Learn more about configuring OpenAPI at https://aka.ms/aspnet/openapi
builder.Services.AddOpenApi();

var app = builder.Build();

app.MapDefaultEndpoints();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.MapOpenApi();
}

app.UseHttpsRedirection();

var myLocations = new[]
{
    "City1", "City2", "City3", "City4", "City5", "City6", "City7", "City8"
};

app.MapGet("/mylocations", (ILoggerFactory loggerFactory) =>
{
    var logger = loggerFactory.CreateLogger("ApiLogger");
    logger.LogInformation("Getting my locations");

    var forecast =  Enumerable.Range(1, 5).Select(index =>
        new WeatherForecast
        (
            DateOnly.FromDateTime(DateTime.Now.AddDays(index)),
            Random.Shared.Next(-20, 55),
            myLocations[Random.Shared.Next(myLocations.Length)]
        ))
        .ToArray();
    return forecast;
})
.WithName("GetMyLocations")
.WithOpenApi();





app.MapGet("/weatherforecasts", (ILoggerFactory loggerFactory) =>
{
    var logger = loggerFactory.CreateLogger("ApiLogger");
    logger.LogInformation("Getting weatherforecasts");

    var forecast = Enumerable.Range(1, 5).Select(index =>
        new WeatherForecast
        (
            DateOnly.FromDateTime(DateTime.Now.AddDays(index)),
            Random.Shared.Next(-20, 55),
            myLocations[Random.Shared.Next(myLocations.Length)]
        ))
        .ToArray();
    return forecast;
})
.WithName("weatherforecasts")
.WithOpenApi();

app.MapHealthChecks("/api-health");

app.Run();

record WeatherForecast(DateOnly Date, int TemperatureC, string? Summary)
{
    public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);
}

AppSettings.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "Serilog": {
    "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.File", "Serilog.Enrichers.Thread", "Serilog.Enrichers.Process", "Serilog.Expressions" , "Serilog.Sinks.ApplicationInsights" ],
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Warning",
        "System": "Warning"
      }
    },
    "Filter": [
      {
        "Name": "ByExcluding",
        "Args": {
          "expression": "RequestPath like '/weatherforecasts%'"
        }
      }
    ],
    "WriteTo": [
      {
        "Name": "Console",
        "Args": {
          "outputTemplate": "[{Timestamp:yyyy:MM:dd hh:mm:ss} {CorrelationId} {Level:u3}] {Message:lj}{NewLine}{Exception}",              
          "expressionTemplate": "{ {@t, @mt, @l: if @l = 'Information' then undefined() else @l, @x, ..@p} }\n",
          "restrictedToMinimumLevel": "Debug"
        }
      },
{
  "Name": "ApplicationInsights",
  "Args": {
    //"connectionString": "[your connection string here]",
    "telemetryConverter": "Serilog.Sinks.ApplicationInsights.TelemetryConverters.TraceTelemetryConverter, Serilog.Sinks.ApplicationInsights"
  }
},
      {
        "Name": "File",
        "Args": {
          "path": "Logs/Apilog-.log",
          "rollingInterval": "Day",
          "retainedFileCountLimit": 31,
          "buffered": false,
          "outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff} [{Level:u3}] {Properties} {Message:lj} {NewLine}{Exception}",
          "expressionTemplate": "{ {@t, @mt, @l, @x, ..@p} }\n",
          "restrictedToMinimumLevel": "Debug"
        }
      }
    ]
  },   
  "AllowedHosts": "*"
}

After running this example project, try to send GET requests to both of the endpoints. This will log only https://localhost:{port}/mylocations endpoint logging into file/console. It will not logged any requests to the https://localhost:{port}/weatherforecasts endpoint.

Output from log file without Serilog filter expression

2025-02-27 19:34:58.641 [INF] { SourceContext: "ApiLogger", RequestId: "0HNAO0K9R9QC3:00000001", RequestPath: "/mylocations/", ConnectionId: "0HNAO0K9R9QC3", Application: "MyApp.Api", Environment: "Development", Version: "1.0.0", MachineName: "WIN0PC" } Getting my locations 
2025-02-27 19:35:00.909 [INF] { SourceContext: "ApiLogger", RequestId: "0HNAO0K9R9QC3:00000002", RequestPath: "/weatherforecasts/", ConnectionId: "0HNAO0K9R9QC3", Application: "MyApp.Api", Environment: "Development", Version: "1.0.0", MachineName: "WIN0PC" } Getting weatherforecasts 
2025-02-27 19:35:02.825 [INF] { SourceContext: "ApiLogger", RequestId: "0HNAO0K9R9QC3:00000003", RequestPath: "/weatherforecasts/", ConnectionId: "0HNAO0K9R9QC3", Application: "MyApp.Api", Environment: "Development", Version: "1.0.0", MachineName: "WIN0PC" } Getting weatherforecasts 
2025-02-27 19:35:04.890 [INF] { SourceContext: "ApiLogger", RequestId: "0HNAO0K9R9QC3:00000004", RequestPath: "/mylocations/", ConnectionId: "0HNAO0K9R9QC3", Application: "MyApp.Api", Environment: "Development", Version: "1.0.0", MachineName: "WIN0PC" } Getting my locations 

Output from log file with Serilog filter expression

2025-02-27 19:36:47.595 [INF] { SourceContext: "ApiLogger", RequestId: "0HNAO0LAA6360:00000001", RequestPath: "/mylocations/", ConnectionId: "0HNAO0LAA6360", Application: "MyApp.Api", Environment: "Development", Version: "1.0.0", MachineName: "WIN0PC" } Getting my locations  
2025-02-27 19:37:08.932 [INF] { SourceContext: "ApiLogger", RequestId: "0HNAO0LAA6360:00000004", RequestPath: "/mylocations/", ConnectionId: "0HNAO0LAA6360", Application: "MyApp.Api", Environment: "Development", Version: "1.0.0", MachineName: "WIN0PC" } Getting my locations

For more information visit this Serilog Expression GitHub repo.

Upvotes: 0

Jason Pan
Jason Pan

Reputation: 22082

The format is not correct, you can use below settings.

{
  "Serilog": {
    "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.ApplicationInsights", "Serilog.Expressions" ],
    "MinimumLevel": {
      "Default": "Debug",
      "Override": {
        "Microsoft.AspNetCore.Mvc.ModelBinding": "Warning",
        "Microsoft.AspNetCore.Routing": "Warning",
        "Microsoft.AspNetCore.Cors.Infrastructure.CorsService": "Warning",
        "Microsoft.EntityFrameworkCore.Infrastructure": "Warning"            
      }
    },
    "WriteTo": [
      {
        "Name": "Logger", 
        "Args": {
          "configureLogger": {
            "Filter": [
              {
                "Name": "ByExcluding",
                "Args": {
                  "expression": "IsDefined(RequestPath)" 
                }
              }
            ],
            "WriteTo": [
              { 
                "Name": "Console" 
              },
              {
                "Name": "ApplicationInsights",
                "Args": {
                  "connectionString": "InstrumentationKey=7aa4cc33-fb03-4b4c-81b8-7a257c083a6e;...",
                  "telemetryConverter": "Serilog.Sinks.ApplicationInsights.TelemetryConverters.TraceTelemetryConverter, Serilog.Sinks.ApplicationInsights"
                }
              }
            ]
          }
        }
      }
    ],
    "Enrich": [ "FromLogContext" ]
  }
}

Upvotes: 0

Related Questions