CODEWITHSUNDEEP
apiresttimeout
Leandro
Leandro

Reputation: 182

Which timeout should I set to an external service?

This service is a remote session pool. I need to ask for a session to work with other services. In most cases, this pool will have a session available, so in 15ms i will have a response. But sometimes, it will need to create a session on demand, requiring up to 800ms to create it.

I have two options in mind to handle this situation:

  1. To set a 15ms timeout, and to implement a retry policy up with an exponential back off until 800ms. This service will create the required session no matter whether I am connected to it.

  2. To set a 800ms timeout, and to keep connected to the service until a session is available for me.

In both cases, there's no guarantee that I will have a session after 800ms.

So the question is: Which are the pro/cons for each option?

Upvotes: 4

Views: 562

Answers (1)

Jonathan Barbero
Jonathan Barbero

Reputation: 2554

1 . To set a 15ms timeout, and to implement a retry policy up with an exponential back off until 800ms. This service will create the required session no matter whether I am connected to it.

Pro

  1. Detects that the session is not available immediately, don't need to wait almost a second for this.
  2. It's up to the client to request again for the session or go by other way, you have more flexibility for different use cases.
  3. You could differentiate the undesired event of waiting for a session more than 15msec reporting each time the fallback strategy goes on, useful for abnormal session pool behaviour detection.

Cons

  1. The code is more complex because of the fallback behaviour.
  2. Multiple parameters because of different timeouts.

2 . To set a 800ms timeout, and to keep connected to the service until a session is available for me.

Pro

  1. Simple and straight-forward implementation
  2. Simple parametrization

Cons

  1. You can't notice the session creation event delay from the session pool. This is important for tracing and diagnostics, this simple approach could hide session pool problems.
  2. Not flexible implementation for different clients use cases.

-

I think the decision driver is if you need a solution that just works for this use case or if this approach will be used for different clients and use cases.

PS: If you need to create a solution for different clients maybe will be worth to create a more complex protocol, like:

// just takes a session if available, no more than 15msec delay expected
get_session(...)  : session 

// if not available, creates one
get_session_or_create(...) : session 

available_sessions(...) : int

//  between 0 and 1, the proportion of available sessions
availability(...) : double  

...

It's up to the client how to use it.

And over dimension the timeout parameters by some safe %, depending on the session creation delay variance.

Upvotes: 1

Related Questions