CODEWITHSUNDEEP
c#.netdesign-patternspollycircuit-breaker
user20358
user20358

Reputation: 14756

Is using a circuit breaker approach to handling http 429 responses too much of an overkill

I have a service that I call. Lets call that LookupService. I have the following line of code in my client application call that service 

var resp= serviceClient.Lookup(payLoad);

Now the LookupService has been configured by the owners of that service to not allow more than 5 concurrent service calls per client. Anymore than that and they will return HTTP 429.

I am thinking of injecting this one call to a checking method just before the call to the Lookup method. Something to the effect of:

ICircuitBreaker cb = CircuitBreakerFactory.GetInstance(); //return a retry instance
cb.Break();
var resp= serviceClient.Lookup(payLoad);

As of right now I want to only pause and retry if the max number of concurrent calls has been hit or if service returns http code 429. I will register to an internal static variable just before each call to the Lookup method and deregister the call when the service has returned. But later on, I will want to implement another class that provides an alternate path of action or uses a fallback service. I am aware there is a retry pattern, but I feel registering each call and keeping track of calls being made to compare against the total allowed calls that the service will allow since I already know that in advance will help me preemptively break the circuit when I know it is going to fail.

Thoughts on this pattern. Workable, perfect, overkill?

Upvotes: 3

Views: 1950

Answers (1)

Peter Csala
Peter Csala

Reputation: 22829

Based on the question that you have raised I can see some confusion around different resilient strategies. Let me try to make them clear.

There are consumer-side (reactive) and producer-side (proactive) patterns / techniques / strategies. The former ones usually react on a fault and the later ones try to prevent the error / fault.

Well-known consumer-side patterns: timeout, retry and circuit breaker.
Well-known producer-side patterns: rate limiting, load shedding, back pressure and bulkhead.

In order to deliver reliable solutions usually is not enough to apply patterns only on one side. In other words, only smart clients can react on servers' back off requests.

According to my understanding you want to limit the concurrent requests from the same machine. Let examine the producer- and consumer-side one-by-one:

1) Producer-side
The rate limiting pattern (also known as throttling) is usually used to prevent request flooding. You can define several pairs of duration and threshold to describe the acceptable load from a given machine:

  • 5 requests / min
  • 100 requests / hour
  • 1000 requests / day

If the user exceeds any of thresholds then you can impose some sort of penalty. Usually this is done by rejecting the requests with 403. When the duration expired then the counter is set back to zero and the service receives requests again from the client.

There are some implementations where the 429 (Too many requests) status code is used instead of 403. This is also correct, but it is a combination of two patterns. The back pressure is the other pattern. This technique can be used to tell gently to the clients that "“I’m currently busy, please come back later". Here you have the ability to define when the client should come back by specifying the Retry-After header. This can be either an absolute or a relative datetime:

  • absolute, for example: Wed, 29 Apr 2020 08:11:00 GMT
  • relative, for example: 120 seconds

2) Consumer-side
On the client side you can use the combination of circuit breaker and retry to achieve the required resilient solution.

The circuit breaker can be used to prevent to flood the temporary inaccessible downstream system. In our case this means whenever the rate limiting exposes the penalty then the breaker can go from Closed to Open state. This will prevent the consumer to send requests to the downstream system and instead it fails immediately on the client side.

The Retry-After header can be used to set the state from Open to Half-Open. So, when the exposed penalty elapsed then the breaker allow to the consumer to make new requests.

In order to avoid loosing any requests, we can apply the retry logic. The retry logic should be aware of the circuit breaker. So, it tries to send a request and if the concurrent requests count is under the threshold then nothing special happens. If the threshold is exceeded then the breaker is Open, so the request will fail immediately and a new attempt is required a bit later. The retry itself can also take advantage of the Retry-After header as well to specify the delay between the initial call and the next attempt.

Please visit the Polly project, where you can find most of these techniques are implemented for .NET developers. I hope this helped you a bit.

Upvotes: 3

Related Questions