Spring Boot Async Timeout Reverting to 30 Seconds Despite Configuration When Using Concurrency

Question

I'm facing a persistent issue in my Spring Boot application where the async timeout reverts to 30 seconds whenever I introduce concurrency using flatMap, parallel(), Schedulers, or any async/parallel execution method. The expected timeout is set to 5 minutes across multiple configurations, but it only works as intended when the execution is sequential.

I'm working with a GraphQL resolver that queries a database in batches. The code works as expected with a 5-minute timeout when the queries are processed sequentially. However, as soon as I introduce parallelism or concurrency, the requests time out after 30 seconds. Here's the relevant code snippet:

@SchemaMapping(typeName = "Items", field = "somePricing")
public Mono<SomePricingResponse> getSomePricing(
       @Argument SomePricingRequest request,
       @Argument Integer pageNo,
       @Argument Integer pageSize) {
   RequestParams params = inputValidator.constructAndValidateRequestParams(pageNo, pageSize);
   inputValidator.validateSomePricingRequest(request);
   return pricingService.getSomePricing(request, params)
           .doOnSuccess(response -> LoggingUtil.logInfo("Fetched Some Pricing"));
}

private Mono<SomePricingResponse> handleBatchingForSomeCombinations(SomePricingRequest request, RequestParams params) {
   List<SomeCombination> allCombinations = request.getSomeCombinations();
   int batchSize = 200;

   // Split the combinations into batches
   List<List<SomeCombination>> batches = new ArrayList<>();
   for (int i = 0; i < allCombinations.size(); i += batchSize) {
       batches.add(allCombinations.subList(i, Math.min(i + batchSize, allCombinations.size())));
   }

   LoggingUtil.logInfo("Total number of batches: {}", batches.size());

   return Flux.fromIterable(batches)
           .timeout(Duration.ofMinutes(5))
           .parallel(10)
           .runOn(Schedulers.newParallel("batch-processing", 10))
           .flatMap(batch -> {
               SomePricingRequest batchRequest = new SomePricingRequest();
               batchRequest.setSomeCombinations(batch);

               int batchNum = batches.indexOf(batch) + 1;
               LoggingUtil.logInfo("Processing batch number {} with {} combinations", batchNum, batch.size());

               // Execute the batch query without a timeout
               return pricingDataAccessLayer.getSomePricing(batchRequest, params)
                       .doOnSuccess(response -> LoggingUtil.logInfo("Batch number {} processed successfully", batchNum));
           }) // Control concurrency here
           .sequential()
           .collectList()
           .map(responses -> {
               SomePricingResponse finalResponse = new SomePricingResponse();
               finalResponse.setData(new ArrayList<>());

               for (SomePricingResponse response : responses) {
                   if (response != null && response.getData() != null) {
                       finalResponse.getData().addAll(response.getData());
                   } else {
                       LoggingUtil.logWarn("One of the batch responses was null or contained null data");
                   }
               }

               finalResponse.setPageable(new Pageable((long) finalResponse.getData().size(), 1L, finalResponse.getData().size(), 0));
               LoggingUtil.logInfo("All batches processed successfully with a total of {} records", finalResponse.getData().size());
               return finalResponse;
           });
}

I've tried using these configurations:

spring:
  mvc:
    async:
      request-timeout: 600000  # 10 minutes
server:
  tomcat:
    connection-timeout: 600000  # 10 minutes
    async-timeout: 600000  # 10 minutes
  async:
    request-timeout: 300000  # 5 minutes
  servlet:
    async:
      timeout: 300000  # 5 minutes
graphql:
  servlet:
    asyncTimeout: 600000

and the logs confirms that it times out after 30 seconds:

2024-08-15 14:22:17.044 [DEBUG] [nio-8080-exec-9] o.s.w.c.request.async.WebAsyncManager: Started async request for "/aais-platform-gql/graphql"
2024-08-15 14:22:17.045 [DEBUG] [nio-8080-exec-9] o.s.web.servlet.DispatcherServlet: Exiting but response remains open for further handling
2024-08-15 14:22:47.153 [DEBUG] [io-8080-exec-10] o.s.w.c.request.async.WebAsyncManager: Servlet container timeout notification for "/aais-platform-gql/graphql"
2024-08-15 14:22:47.154 [DEBUG] [io-8080-exec-10] o.s.w.c.request.async.WebAsyncManager: Async result set to: org.springframework.web.context.request.async.AsyncRequestTimeoutException for "/aais-platform-gql/graphql"

I’m looking for guidance on:

Identifying if there’s a hidden configuration or external service that could be overriding my timeout settings. Any alternative approaches to handling concurrency that might bypass this 30-second limit. Insights into whether there could be an issue with my application’s interaction with the Tomcat server or Spring’s async handling. Any help or pointers would be greatly appreciated!

Answer 1

the Tomcat Async timeout value couldn't be set in the config, but I only was able to set it explicitly like this:

@Configuration
public class TomcatConfig {

   @Bean
   public TomcatServletWebServerFactory tomcatServletWebServerFactory() {
       TomcatServletWebServerFactory factory = new TomcatServletWebServerFactory();
       factory.addConnectorCustomizers(connector -> connector.setAsyncTimeout(600000)); // Set async timeout to 10 minutes
       return factory;
   }
}

This effectively increased the default timeout from 30 seconds to 5 minutes.