CODEWITHSUNDEEP
c#.netrate-limitingpollyretry-logic
Daan
Daan

Reputation: 6994

How to configure the Rate Limiter of the StandardResilienceHandler

I'm writing an application that connects to another HTTP service. I want to rate-limit my outgoing requests to that external service.

Ideally, I want to use the standard resilience handler as documented here, to benefit from the configured best-practice defaults when it comes to resiliency.

services.AddHttpClient<ExternalServiceGateway>()
    .AddStandardResilienceHandler();

However, since I know that the external service is rate-limited to 120 req/sec, I want to apply a similar policy to my outgoing calls.

I can achieve this using a separately-configured ResiliencePipeline (see below), but I want to combine this with the AddStandardResilienceHandler(). Is there any way to achieve this?

var pipeline = new ResiliencePipelineBuilder()
    .AddRetry(new RetryStrategyOptions
    {
        ShouldHandle = new PredicateBuilder().Handle<RateLimiterRejectedException>(),
        Delay = TimeSpan.FromSeconds(1),
        MaxRetryAttempts = 5,
        BackoffType = DelayBackoffType.Exponential,
        UseJitter = true
    })
    .AddRateLimiter(new FixedWindowRateLimiter(new FixedWindowRateLimiterOptions()
    {
       PermitLimit = 2,
       Window = TimeSpan.FromSeconds(1)
    }))
    .Build();

Update: example code for problem/solution

I have published a minimal solution to experiment with this problem/solution here.

Upvotes: 1

Views: 154

Answers (1)

Peter Csala
Peter Csala

Reputation: 22679

The AddStandardResilienceHandler has an overload where you can override each of the predifined strategies' options. This can be as simple as this:

.AddStandardResilienceHandler(options =>
{
    options.AttemptTimeout.Timeout = TimeSpan.FromSeconds(60);
});

The rate limiter override is a bit tricky. You have to acquire a lease like this:

.AddStandardResilienceHandler(options => 
{ 
    options.RateLimiter.RateLimiter = args => 
    {
        var limiter = new FixedWindowRateLimiter(new FixedWindowRateLimiterOptions()
        {
            PermitLimit = 2,
            Window = TimeSpan.FromSeconds(1)
        });
        return limiter.AcquireAsync(cancellationToken: args.Context.CancellationToken);
    };
});

Update #1

As it was discussed in the comments section the above suggested solution does not work as expected. The main reason is that a new RateLimiter is instantiated everytime. That's why the limit is never reached. The fix for this is quite simple. We have to define the ratelimiter outside of the RateLimiter delegate

.AddStandardResilienceHandler(options => 
{ 
    var limiter = new FixedWindowRateLimiter(new FixedWindowRateLimiterOptions()
    {
        PermitLimit = 2,
        Window = TimeSpan.FromSeconds(1)
    });
    options.RateLimiter.RateLimiter = args 
       => limiter.AcquireAsync(cancellationToken: args.Context.CancellationToken);

});

Update #2

the next challenge is that the RateLimiterRejectedException that is now thrown when the policy is triggered, is not automatically picked up and 'handled' by the retry policy.

This Microsoft Learn material says the following:

The retry pipeline retries the request in case the dependency is slow or returns a transient error.

And in the very next section it details the handled exceptions:

  • HttpRequestException
  • TimeoutRejectedException

If you are interested here you can find the related source code.

So, how can we add RateLimiterRejectedException to the current predicate? The ShouldHandle outcome is a ValueTask<bool> so if we await it then we can simple add || args.Outcome.Exception is RateLimiterRejectedException:

.AddStandardResilienceHandler(options => 
{ 
    // Rate limiter
    var limiter = new FixedWindowRateLimiter(new FixedWindowRateLimiterOptions()
    {
        PermitLimit = 2,
        Window = TimeSpan.FromSeconds(1)
    });
    options.RateLimiter.RateLimiter = args 
       => limiter.AcquireAsync(cancellationToken: args.Context.CancellationToken);

    // Retry
    var standardPredicate = options.Retry.ShouldHandle;
    options.Retry.ShouldHandle = async args => await standardPredicate(args) 
       || args.Outcome.Exception is RateLimiterRejectedException;

});

Please make sure that you don't inline the standardPredicate(args). That would make the ShouldHandle recursive and it will shortly fail with a Stack overflow.

Here you can play with a simplified dotnet fiddle application.

Update #3

I was playing a bit with your code when I realized for that you can't use the standard resilience handler to retry for ratelimiter because retry is an inner strategy from the rate limiter perspective.

public static IHttpStandardResiliencePipelineBuilder AddStandardResilienceHandler(this IHttpClientBuilder builder)
{
    ...
    _ = builder.AddResilienceHandler(StandardIdentifier, (builder, context) =>
    {
        ...
        _ = builder
            .AddRateLimiter(options.RateLimiter)
            .AddTimeout(options.TotalRequestTimeout)
            .AddRetry(options.Retry)
            .AddCircuitBreaker(options.CircuitBreaker)
            .AddTimeout(options.AttemptTimeout);
    });
}

That means the rate limiter is the outer most strategy. If every other strategy failed to handle request then it will reach the rate limiter. That's why changing the retry strategy's ShouldHandle caused no retries.

Here we have documented that the strategies registration order does matter.

Upvotes: 3

Related Questions