Polly Retries Doesn't Close Existing HttpClient Call

Question

I'm using Polly to handle some scenarios like request throttled and timeouts. The policies were added directly in the Startup.cs which would be like this :

var retries = //applying the retries, let say I set to 25 times with 10s delay. Total 250s.

serviceCollection
    .AddHttpClient<IApplicationApi, ApplicationApi>()
    .AddPolicyHandler((services, request) => GetRetryPolicy<ApplicationApi>(retries, services));

The Policy:

static IAsyncPolicy<HttpResponseMessage> GetRetryPolicy<T>(List<TimeSpan> retries, IServiceProvider services)
{
    return HttpPolicyExtensions
        .HandleTransientHttpError()
        .OrResult(msg => msg.StatusCode == System.Net.HttpStatusCode.TooManyRequests)
        .WaitAndRetryAsync(retries,
            onRetry: (outcome, timespan, retryAttempt, context) =>
            {
                //do some logging
            }
}

In ApplicationApi.cs do something like this:

private readonly HttpClient _httpClient;
public ApplicationApi(HttpClient httpClient)
{
    _httpClient = httpClient;
}
       
public void CallApi()
{ 
      var url = "https://whateverurl.com/someapi"
      using (var request = new HttpRequestMessage(HttpMethod.Get, url))
      {
          var response = await _httpClient.SendAsync(request);
          var respMessage = await 
          response.Content.ReadAsStringAsync();
      }
}

Now let say I don't specify the HttpClient.Timeout, which then will use default timeout : 100s.

Now I have a problem with heavy throttling. Polly will retry until the throttling resolved, or it reach the max retry. But, the program will thrown an exception on the 10th retry since it already more than 100s elapsed on httpclient since the first request it got throttled.

Seems like the first http request that got throttled still on and not closed, or I may be wrong. What causing this? Is it a normal behavior of Polly retries? How can I make it close the connection on each retries so I don't have to set a very high HttpClient.Timeout value.

I also implemented the Polly timeout policy to cut request if more than some specified second then retry until it succeed. But the Polly behavior still like this. So I need to set httpclient timeout > total elapsed time on retries

**UPDATE Code updated. So I just realized there's using statement for the request.

***UPDATE I've created a repo that reproduce the behavior here : https://github.com/purnadika/PollyTestWebApi

Answer 1

The short answer is that your observed behaviour is due to fact how AddPolicyHandler and PolicyHttpMessageHandler work.

Whenever you register a new Typed HttpClient without any policy (.AddHttpClient) then you basically create a new HttpClient like this:

var handler = new HttpClientHandler();
var client = new HttpClient(handler);

Of course it is much more complicated, but from our topic perspective it works like that.

If you register a new Typed HttpClient with a policy (.AddHttpClient().AddPolicyHandler()) then you create a new HttpClient like this

var handler = new PolicyHttpMessageHandler(yourPolicy);
handler.InnerHandler = new HttpClientHandler();
var client = new HttpClient(handler);

So the outer handler will be the Polly's MessageHandler and the inner is the default ClientHandler.

Polly's MessageHandler has the following documentation comment:

/// <para>
/// Take care when using policies such as Retry or Timeout together as HttpClient provides its own timeout via
/// <see cref="HttpClient.Timeout"/>.  When combining Retry and Timeout, <see cref="HttpClient.Timeout"/> will act as a
/// timeout across all tries; a Polly Timeout policy can be configured after a Retry policy in the configuration sequence,
/// to provide a timeout-per-try.
/// </para>

By using the AddPolicyHandler the HttpClient's Timeout will act as a global timeout.

---

The solution

There is workaround, namely avoiding the usage of AddPolicyHandler.

So, rather than decorating your Typed Client at the registration time you can decorate only the specific HttpClient method call inside your typed client.

Here is a simplified example based on your dummy project:

• ConfigureServices

services.AddHttpClient<IApplicationApi, ApplicationApi>(client => client.Timeout = TimeSpan.FromSeconds(whateverLowValue));

• _MainRequest

var response = await GetRetryPolicy().ExecuteAsync(async () => await _httpClient.GetAsync(url));

Here I would like to emphasize that you should prefer GetAsync over SendAsync since the HttpRequestMessage can not be reused.

So, if you would write the above code like this

using (var request = new HttpRequestMessage(HttpMethod.Get, url))
{
   var response = await GetRetryPolicy().ExecuteAsync(async () => await _httpClient.SendAsync(request));
}

then you would receive the following exception:

InvalidOperationException: The request message was already sent. Cannot send the same request message multiple times.

So, with this workaround the HttpClient's Timeout will not act as a global / overarching timeout over the retry attempts.

Answer 2

Polly works to retry the top-level HttpClient request, so HttpClient's timeout applies to all retries. That's the whole point of using Polly, to retry requests in a way that's transparent to the top-level code.

If retrying for over a minute failed to work, the retry policy isn't good enough. Retrying over and over with a fixed delay will only result in more 429 responses, as all the requests that failed will be retried at the same time. This will result in wave after wave of identical requests hitting the server, resulting in 429s once again.

To avoid this, exponential backoff and jitter are used to introduce an increasing random delay to retries.

From the linked sample:

var delay = Backoff.DecorrelatedJitterBackoffV2(
        medianFirstRetryDelay: TimeSpan.FromSeconds(1), 
        retryCount: 5);

var retryPolicy = Policy
    .Handle<FooException>()
    .WaitAndRetryAsync(delay);

The Polly page for the Jitter strategy explain how this works. The distribution graph of delays shows that even with 5 retries, the retry intervals don't clamp together.

This means there's less chance of multiple HttpClient calls retrying at the same time, resulting in renewed throttling