CODEWITHSUNDEEP
asp.net-core.net-coreserilogserilog-aspnetcoreserilog-sinks-async
lonix
lonix

Reputation: 21011

Serilog async sink with health monitoring

The Serilog async sink has a health monitoring feature, where one must implement IAsyncLogEventSinkMonitor to monitor when the buffer is overwhelmed and log events are dropped.

The sample shown is pseudocode without much detail. I'm unsure how to implement it.

Upvotes: 1

Views: 73

Answers (1)

lonix
lonix

Reputation: 21011

Here's a simple demo implementation, based on various issues in the repo ([1], [2]).

AsyncSinkMonitor.cs

using Serilog.Debugging;
using Serilog.Sinks.Async;

namespace Demo;

public sealed class AsyncSinkMonitor : IAsyncLogEventSinkMonitor, IDisposable
{

  public static AsyncSinkMonitor Instance { get; } = new();

  private const int TIMER_PERIOD_MILLISECONDS = 250;    // choose carefully; too low would cause perf drop, too high would miss dropped logs
  private const double THRESHOLD_WARN = .5;
  private const double THRESHOLD_CRIT = .9;
  private System.Timers.Timer? _timer;

  public void Dispose()
  {
    if (_timer == null) return;
    _timer.Stop();
    _timer.Dispose();
    _timer = null;
  }

  public void StartMonitoring(IAsyncLogEventSinkInspector inspector)
  {
    _timer?.Dispose();
    _timer = new System.Timers.Timer(TIMER_PERIOD_MILLISECONDS);
    _timer.Elapsed += (sender, args) => OnTimerElapsed(inspector);
    _timer.Start();
  }

  public void StopMonitoring(IAsyncLogEventSinkInspector inspector) =>
    Dispose();

  private void OnTimerElapsed(IAsyncLogEventSinkInspector inspector)
  {
    var utilisation = (double)inspector.Count / inspector.BufferSize;
    var nDrops      = inspector.DroppedMessagesCount;    // cumulative value; is never reset to zero

         if (THRESHOLD_WARN < utilisation) SelfLog.WriteLine("WARN: Async buffer exceeded {0:p0} utilisation; will drop log events when reaches 100%; dropped {1} since app start", utilisation, nDrops);
    else if (THRESHOLD_CRIT < utilisation) SelfLog.WriteLine("CRIT: Async buffer exceeded {0:p0} utilisation; will drop log events when reaches 100%; dropped {1} since app start", utilisation, nDrops);
  }

}

appsettings.json

"WriteTo": [
  {
    "Name": "Async",
    "Args": {
      "bufferSize": 10000,    // 10k is default
      "blockWhenFull": "false",
      "monitor": "Demo.AsyncSinkMonitor::Instance, Demo",
      "configure": [{
        // wrapped sink...
      }]
    }
  }
]

Or in code:

.WriteTo.Async(
  x => x.WrappedSink(),
  bufferSize: 10_000,
  blockWhenFull: false,
  monitor: AsyncSinkMonitor.Instance
)

Notes:

  • It writes to Serilog's "self log", but one could write to a file or elsewhere
  • Choose the timer's period carefully: too low would cause a perf drop, too high would miss dropped logs; intuitively, a value in the range 250..500ms is probably fine
  • Even without above code, the background worker logs to the self log anyway, as follows:

    2024-11-20T08:13:47.7427088Z Serilog.Sinks.Async.BackgroundWorkerSink: unable to enqueue, capacity 10000 (Permanent, 1 events)

Upvotes: 1

Related Questions