Limiting rate of requests with Reactor

Question

I'm using project reactor to load data from a web service using rest. This is done in parallel with multiple threads. I'm starting to hit rate limits on the web service, so I would like to send at most 10 requests per second to avoid getting these errors. How would I do that using reactor?

Using zipWith(Mono.delayMillis(100))? Or is there some better way?

Thank you

Answer 1

One could use Flux.delayElements to process a 10 requests batch at every 1s; be aware though that if the processing takes longer than 1s the next batch will still be started in parallel hence being processed together with the previous one (and potentially many other previous ones)!

That's why I propose another solution where a 10 requests batch is still processed at every 1s but, if its processing takes longer than 1s, the next batch will fail (see overflow IllegalStateException); one could deal with that failure such that to continue the overall processing but I won't show that here because I want to keep the example simple; see onErrorResume useful to handle overflow IllegalStateException.

The code below will do a GET on https://www.google.com/ at a rate of 10 requests per second. You'll have to do additional changes in order to support the situation where your server is not able to process in 1s all your 10 requests; you could just skip sending requests when those asked at previous second are still processed by your server.

@Test
void parallelHttpRequests() {
    // this is just for limiting the test running period otherwise you don't need it
    int COUNT = 2;

    // use whatever (blocking) http client you desire;
    // when using e.g. WebClient (Spring, non blocking client)
    // the example will slightly change for no longer use
    // subscribeOn(Schedulers.elastic())
    RestTemplate client = new RestTemplate();
    
    // exit, lock, condition are provided to allow one to run 
    // all this code in a @Test, otherwise they won't be needed
    var exit = new AtomicBoolean(false);
    var lock = new ReentrantLock();
    var condition = lock.newCondition();

    MessageFormat message = new MessageFormat("#batch: {0}, #req: {1}, resultLength: {2}");
    Flux.interval(Duration.ofSeconds(1L))
            .take(COUNT) // this is just for limiting the test running period otherwise you don't need it
            .doOnNext(batch -> debug("#batch", batch)) // just for debugging
            .flatMap(batch -> Flux.range(1, 10) // 10 requests per 1 second
                            .flatMap(i -> Mono.fromSupplier(() ->
                                    client.getForEntity("https://www.google.com/", String.class).getBody()) // your request goes here (1 of 10)
                                    .map(s -> message.format(new Object[]{batch, i, s.length()})) // here the request's result will be the output of message.format(...)
                                    .doOnSubscribe(s -> debug("doOnSubscribe: #batch = " + batch + ", i = " + i)) // just for debugging
                                    .subscribeOn(Schedulers.elastic()) // one I/O thread per request
                            )
            )
            // consider using onErrorResume to handle overflow IllegalStateException
            .subscribe(
                    s -> debug("received", s) // do something with the above request's result
                    e -> {
                        // pay special attention to overflow IllegalStateException
                        debug("error", e.getMessage());
                        signalAll(exit, condition, lock);
                    },
                    () -> {
                        debug("done");
                        signalAll(exit, condition, lock);
                    }
            );

    await(exit, condition, lock);
}

// you won't need the "await" and "signalAll" methods below which
// I created only to be easier for one to run this in a test class

private void await(AtomicBoolean exit, Condition condition, Lock lock) {
    lock.lock();
    while (!exit.get()) {
        try {
            condition.await();
        } catch (InterruptedException e) {
            // maybe spurious wakeup
            e.printStackTrace();
        }
    }
    lock.unlock();
    debug("exit");
}

private void signalAll(AtomicBoolean exit, Condition condition, Lock lock) {
    exit.set(true);
    try {
        lock.lock();
        condition.signalAll();
    } finally {
        lock.unlock();
    }
}

Answer 2

You can use delayElements instead of the whole zipwith.