CODEWITHSUNDEEP
javaguava
FidelCashflo
FidelCashflo

Reputation: 533

Guava RateLimiter warmup clarification

I'm trying to figure out a way to calculate how many QPS will happen at a certain time given that I am using Guava ratelimiter with warmpup. I read the comments here, but it's still unclear to me. Hopefully someone here can clarify for me.

My use case:

I call an external service which has a 50 TPS limit. The caveat is, the first ~500 times we call them must be well below the 50 TPS afterwards we can resume 50TPS. (If there is a better solution than using the ratelimiter, I'd love to hear it!)

Psuedocode:

RateLimiter rateLimiter = RateLimiter.create(50.0, 10, minutes);
for (String customerId : customerList) {
    rateLimiter.acquire();
    // call external service
}

Assuming we're only doing this with one thread. Is there a way to calculate what the TPS(QPS) will be at a given time? 3 minutes in? 5 minutes in? etc.

Upvotes: 3

Views: 3192

Answers (1)

mfulton26
mfulton26

Reputation: 31234

The cold (minimum) rate of a RateLimiter with a warmupPeriod is 1/3 of the stable (maximum) rate (this is derived from the coldFactor hardcoded to 3.0 in RateLimiter.java:147-184). Under saturated demand (i.e. uninterrupted requests for permits) the rate will increase at a constant rate until the stable (maximum) rate is reached.

As this is a linear equation we can write this in the form y = m * x + b where

  • y (a.k.a y(x)), or qps(t)), is our expected QPS given a saturated period (e.g. 3 minutes in),
  • m is the rate of change from our cold (minimum) rate to our stable (maximum) rate,
  • x (or t) is the elapsed time under saturated demand, and
  • b is our cold (minimum) rate (1/3 of our stable (maximum) rate).

Putting it altogether we have qps(t) = (stableRate - coldRate) / warmupPeriod * saturatedPeriod + coldRate where coldRate = stableRate / 3.

So for your example we can get the expected QPS at 3 minutes in:

qps(3) = (50.0 - 50.0/3.0) / 10.0 * 3.0 + 50.0/3.0 ~= 26.6666

Here is an implementation in Java:

/**
 * Calculates the expected QPS given a stableRate, a warmupPeriod, and a saturatedPeriod.
 * <p>
 * Both specified periods must use the same time unit.
 *
 * @param stableRate      how many permits become available per second once stable
 * @param warmupPeriod    the duration of the period where the {@code RateLimiter} ramps up
 *                        its rate, before reaching its stable (maximum) rate
 * @param saturatedPeriod the duration of the period for which the {@code RateLimiter} has
 *                        been under saturated demand starting from a cold state
 * @return The expected QPS assuming saturated demand starting from a cold state
 */
public static double qps(double stableRate, double warmupPeriod, double saturatedPeriod) {
    if (saturatedPeriod >= warmupPeriod) {
        return stableRate;
    }
    double coldRate = stableRate / 3.0;
    return (stableRate - coldRate) * saturatedPeriod / warmupPeriod + coldRate;
}

Note that in practice using a single thread you will not be able to saturate the demand on a RateLimiter so your actual QPS will be slightly lower than expected and will rarely (if ever) actually reach the stable (maximum) rate. Using multiple threads, however, will allow you to always have a pending permit request and saturate demand.

Upvotes: 4

Related Questions