How to perform request coalescing with .net middleware

Question

Firstly, I know there are options for request coalescing, like using an Origin Shield, Varnish, etc.

But those are architectural changes and I was looking for something more straight forward.

My thought was to use some middleware that would first determine if the request being processed was enabled for request coalescing (an attribute on the endpoint) but my lack of understanding middleware blocked me from this option and reaching out to the community for a middleware solution or an alternate solution (yet remaining in the .net project).

My inital look is at .net core 3.1, but also looking for .net 6/7 support.

The idea would be to determine if a request is a duplicate (using query params, headers), if so, re-use the result from the first request. Therefore only 1 request would get processed for a spike of requests that are the same.

Request coalescing may also be known as request collapsing.

I first created an attribute that I could add an endpoint that would enable it to be coalesed with other similar requests, but I didn't even get far enough to actually used it. For those interested, it was the following code:

public class SquashableAttribute : Attribute
{
    public SquashableAttribute()
    {
    }
}

I created the following middleware (and extension method):

public class SquasherMiddleware
{
    private readonly RequestDelegate _next;

    public SquasherMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        string path = context.Request.Path.Value;
        Console.WriteLine($"CHECKING SQUASHABLE FOR {path}");
        await _next.Invoke(context);
        Console.WriteLine($"SQUASHER FINISHED FOR {path}");
    }
}

public static class SquasherMiddlewareExtensions
{
    public static IApplicationBuilder UseSquasher(this IApplicationBuilder builder) => builder.UseMiddleware<SquasherMiddleware>();
}

And a simple endpoint:

[HttpGet]
[Squashable]
public async Task<IActionResult> GetAsync()
{
    await Task.Run(async () => await Task.Delay(10000));

    return Ok();
}

Hooking up in Startup.cs (after app.UseRouting() so we can get the path in the middleware):

app.UseSquasher();

What I noticed when hitting this endpoint multiple times, I would only see the log "CHECKING SQUASHABLE FOR {path}" in the logs for the first request and nothing for the second request until the first request finished. However, if I made a request to this endpoint and then made the request to a DIFFERENT endpoint, then I saw the log for the second request before the first request had complete.

It seems that the middleware doesn't run for the same request until another completes, but different requests run as expected.

Answer 1

Here is you reworked middleware through Polly collapser:

public class SquashableAttribute : Attribute
{
}

public class SquasherMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger<SquasherMiddleware> _logger;
    private static readonly IAsyncRequestCollapserPolicy CollapserPolicy = AsyncRequestCollapserPolicy.Create();

    public SquasherMiddleware(RequestDelegate next, ILogger<SquasherMiddleware> logger)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var endpoint = context.Features.Get<IEndpointFeature>()?.Endpoint;
        var attribute = endpoint?.Metadata.GetMetadata<SquashableAttribute>();
        if (attribute == null)
        {
            await _next.Invoke(context).ConfigureAwait(false);
        }
        else
        {
            await OnInvokeAsync(context).ConfigureAwait(false);
        }
    }

    private async Task OnInvokeAsync(HttpContext httpContext)
    {
        string path = httpContext.Request.Path.Value;
        var collapserContext = new Context(path);
        collapserContext["http"] = httpContext;
        _logger.LogInformation($"Check if pending: {collapserContext.OperationKey}");
        var collapsedContext = await CollapserPolicy.ExecuteAsync(OnDownstreamInvokeAsync, collapserContext).ConfigureAwait(false);
        if (collapsedContext != httpContext)
        {
            _logger.LogInformation($"Collapse your contexts: {path}");
            //copy past your relevant Body/Headers from collapsedContext to httpContext
        }
    }

    private async Task<HttpContext> OnDownstreamInvokeAsync(Context ctx)
    {
        var httpContext = (HttpContext)ctx["http"];
        _logger.LogInformation($"Executing on downstream: {ctx.OperationKey}");
        await _next.Invoke(httpContext).ConfigureAwait(false);
        return httpContext;
    }
}

Controller:

[ApiController]
[Route("ping")]
public class PingController : ControllerBase
{
    [HttpGet]
    [Squashable]
    public async Task<IActionResult> Get()
    {
        await Task.Delay(10000).ConfigureAwait(false);
        return Ok();
    }
}

Result in logs:

info: WebApplication2.SquasherMiddleware[0]
      Check if pending: /ping
info: WebApplication2.SquasherMiddleware[0]
      Executing on downstream: /ping   <--------- single call to your endpoint
info: WebApplication2.SquasherMiddleware[0]
      Check if pending: /ping          <--------- every other call is waiting
info: WebApplication2.SquasherMiddleware[0]
      Check if pending: /ping
info: WebApplication2.SquasherMiddleware[0]
      Check if pending: /ping
info: WebApplication2.SquasherMiddleware[0]
      Check if pending: /ping
info: WebApplication2.SquasherMiddleware[0]
      Check if pending: /ping
info: WebApplication2.SquasherMiddleware[0]
      Collapse your contexts: /ping    <----- released all of them when first is complete. You are free to use HttpContext to populate all other HttpContexts (with body, headers, etc)
info: WebApplication2.SquasherMiddleware[0]
      Collapse your contexts: /ping
info: WebApplication2.SquasherMiddleware[0]
      Collapse your contexts: /ping
info: WebApplication2.SquasherMiddleware[0]
      Collapse your contexts: /ping
info: WebApplication2.SquasherMiddleware[0]
      Collapse your contexts: /ping