CODEWITHSUNDEEP
javaspring-bootasynchronousjava-21virtual-threads
SHM
SHM

Reputation: 194

Controlling Async Processing Rate in Spring Boot with Virtual Threads

I'm encountering an issue with excessive async processing causing : 429 Too Many Requests errors(async logic is to call 3rd party API) in my Spring Boot application. Here are the relevant details:

Java version: 21
SpringBoot version: 3.2.3
Virtual Thread enabled (spring.threads.virtual.enabled=true)

Currently, I'm utilizing @Async with Executors.newVirtualThreadPerTaskExecutor() to handle asynchronous tasks. However, the unrestricted thread creation under high load leads to the throttling issue.

To mitigate this, I'm considering replacing the executor with a fixedThreadPool. However, I'm seeking advice on more optimal methods to control the rate of async processing.

Below is the relevant code snippet:

@Bean
public AsyncTaskExecutor applicationTaskExecutor() {
   return new TaskExecutorAdapter(Executors.newVirtualThreadPerTaskExecutor());
}

Update: Updated AyncTaskExecutor config

@Bean
public AsyncTaskExecutor applicationTaskExecutor() {
 return new VirtualThreadTaskExecutor("async-worker");
}

Upvotes: 1

Views: 1067

Answers (1)

igor.zh
igor.zh

Reputation: 2348

If you can use TaskExecutorAdapter then you might want to check out this answer. Configuring your own applicationTaskExecutor, you in fact disable the default SimpleAsyncTaskExecutor, which is configured from Spring Boot application properties like spring.threads.virtual.enabled. You might want not do it and let Spring use the default one, if so then you can find helpful this answer.

I think, however, that 429 Too Many Requests HTTP status is of too dynamic and unpredictable nature, and therefore no known off-the-shelf executor is capable of the correspondent dynamic tuning. The better and more natural to virtual threads way is to spawn them directly:

final HttpStatus[] status = new HttpStatus[1];
    
Thread.ofVirtual().start( () -> {
    // call third party, 429-prone service 
    //status[0] = ...
});
    
if (status[0] == HttpStatus.TOO_MANY_REQUESTS) {
    Thread.sleep(SUSPEND_TIME);
}

This snippet assumes that suspending a caller thread is allowed from the standpoint of your application's threading architecture. If it is not, then, depending on your exact logic of handling 429, Circuit Breaker, Rate Limiter, Bulkhead etc patterns implemented by Resilience4j, Hystrix, Spring Retry etc might be your way to go while the simplest homemade solution might be just a single dedicated (daemon platform) thread that implements the snippet above.

I'm considering replacing the executor with a fixedThreadPool

If you mean here Executors.newFixedThreadPool(MAX_THREADS), then you plainly disable the usage of virtual threads by this executor, regardless of any property values.

Upvotes: 1

Related Questions