ITelemetryProcessor uses 5% total CPU - log filtering in dotnet-isolated azure function

Question

I have the following telemetry processor:

public class ExcludeInProcInvokeTelemetryProcessor(ITelemetryProcessor next) : ITelemetryProcessor
{
    private readonly ITelemetryProcessor _next = next;

    public void Process(ITelemetry item)
    {
        if (item is DependencyTelemetry dependency && dependency.Type == "InProc" && dependency.Name == "Invoke")
            return;

        _next.Process(item);
    }
}

Visual studio's diagnostic tools show that this method uses 5% total CPU, I suppose because it's being called for every single piece of telemetry.

Is there another way that I could be doing this filtering? This processor is running in an azure function on the isolated process model, so I can't filter at the host level, if such a thing is possible.

Any other tips for performance optimization here would be greatly appreciated!

In the grand scheme of things, 5% CPU isn't a huge amount, but it does seem unnecessary considering the simplicity of the filter. I have another more complicated filter, that doesn't show up in the top resource-consuming methods of diagnostic tools.

Answer 1

Is there another way that I could be doing this filtering?

By using sampling telemetry to filter some logs in Application Insights for Azure Functions in the isolated process model, I could implemented a custom telemetry processor that applies sampling. check below code:

Function code:

public class Function1
{
    private readonly ILogger _logger;
    private readonly TelemetryClient _telemetryClient;

    public Function1(ILoggerFactory loggerFactory, TelemetryClient telemetryClient)
    {
        _logger = loggerFactory.CreateLogger<Function1>();
        _telemetryClient = telemetryClient;
    }

    [Function("Function1")]
    [Obsolete]
    public void Run([TimerTrigger("0 */5 * * * *")] TimerInfo myTimer)
    {
        _logger.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
        _telemetryClient.TrackEvent("TimerTriggerEvent");
        _telemetryClient.TrackDependency("HTTP", "https://example.com", DateTime.UtcNow, TimeSpan.FromMilliseconds(500), true);

        _telemetryClient.TrackTrace("This is a custom trace message.");
    }
}

Program.cs with Sampling telemetry configuration:

public class Program
{
    public static void Main(string[] args)
    {
        var host = new HostBuilder()
            .ConfigureFunctionsWebApplication()
            .ConfigureServices((context, services) =>
            {
                // Retrieve the Application Insights connection string from environment variables
                var appInsightsConnectionString = context.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];

                // Configure Application Insights telemetry
                services.AddSingleton<TelemetryConfiguration>(sp =>
                {
                    var configuration = new TelemetryConfiguration
                    {
                        ConnectionString = appInsightsConnectionString
                    };

                    // Register the custom sampling telemetry processor
                    configuration.DefaultTelemetrySink.TelemetryProcessorChainBuilder
                        .Use((next) => new SamplingTelemetryProcessor(next, 10)) // 10% sampling
                        .Build();

                    return configuration;
                });

                // Register TelemetryClient
                services.AddSingleton<TelemetryClient>(sp =>
                {
                    return new TelemetryClient(sp.GetRequiredService<TelemetryConfiguration>());
                });
            })
            .Build();

        host.Run();
    }
}

// Custom Sampling Telemetry Processor
public class SamplingTelemetryProcessor : ITelemetryProcessor
{
    private ITelemetryProcessor _next;
    private static Random _random = new Random();
    private double _samplingPercentage;

    public SamplingTelemetryProcessor(ITelemetryProcessor next, double samplingPercentage)
    {
        _next = next;
        _samplingPercentage = samplingPercentage;
    }

    public void Process(ITelemetry item)
    {
        if (_random.NextDouble() * 100 < _samplingPercentage)
        {
            _next.Process(item);
        }
    }
}

local.settings.json:

{
    "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated",
    "APPLICATIONINSIGHTS_CONNECTION_STRING": "your-application-insights connection-string"
  }
}

• The logs in application insights before adding sampling telemetry configuration.

• The logs are filtered in application insights after added the sampling telemetry configuration. check below:

Output: