Spring boot with java 21 virtual threads - is there a way to terminate gracefully?

Question

In SB 3.4 + java 21, when you terminate the java process, regardless of type of task executor, it will wait till all threads finish, which will include scheduled jobs as well as API calls.

However, if you use virtual threads instead of a thread pool by adding this: spring.threads.virtual.enabled=true to the application.properties, and you shut down your application, it appears to terminate immediately, not waiting for any thread to finish.

If so, is there an option to prevent this, and shut down gracefully?

This applies to @scheduled jobs and also web apis.

Answer 1

@Scheduled Jobs

Yes, Spring Scheduling does raise a question of graceful shutdown if virtual threads are enabled by setting spring.threads.virtual.enabled to true. Indeed, in this case the non-pooling virtual thread Executor is used, virtual threads are by default daemons and this cannot be changed, and when the Spring Application Context gets closed, the virtual threads that might be busy doing Scheduler jobs will be immediately and disgracefully terminated.

Java Concurrency does offer graceful shutdown of the previously submitted tasks via ExecutorService.shutdown, shutdownNow, and awaitTermination methods, and ScheduledExecutorService implementations, used by Spring Scheduling, fully support this feature.

But the problem is that all these implementations pool the threads while pooling is not recommended for virtual threads. Therefore, the developers might consider the fact that virtual threads are not very suitable for Spring Scheduling.

One solution to this problem is to turn off virtual threads for Spring Scheduling. For that, it is necessary to implement SchedulingConfigurer and provide custom taskExecutor, the details are discussed in an answer to a thread Does spring @Scheduled annotated methods runs on different threads?:

@Configuration
@EnableScheduling
public class AppConfig implements SchedulingConfigurer {

    @Override
    public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
        taskRegistrar.setScheduler(taskExecutor());
    }

    @Bean
    public Executor taskExecutor() {
        return Executors.newScheduledThreadPool(100);
    }

 } 

The application will still enjoy the virtual threads for other purposes, like servlet container (e.g. Tomcat) worker threads or @Async-annotated methods.

Other solution is built upon an idea that virtual threads still can be pooled. In this case custom taskExecutor in the above snippet may look like follows:

    @Bean
    public Executor taskExecutor() {
        return Executors.newScheduledThreadPool(100, Thread.ofVirtual().factory());
    }

This choice is discussed in my answer to a thread How to migrate from Executors.newFixedThreadPool(MAX_THREAD_COUNT()) to Virtual Thread, and although the topic of the thread is slightly different, the issue of virtual thread pooling applies to the second approach; my answer argues that pooling of virtual threads is harmless, even being senseless for most cases.

Both above solutions guarantee that the Application Context won't be closed until all threads, platform or virtual, used for scheduling, terminate.

Note that in both cases the application code does not need to invoke ExecutorService.shutdown, shutdownNow, and awaitTermination methods, as they would be invoked upon rather complex chain of events, triggered by Application Context closure.

API Endpoints

Things are much better in this case, and no Loom guidelines on virtual threads pooling should be violated.

If default for Spring Boot applications Tomcat is used as web server/servlet container, then the Web API endpoints, for example, REST endpoints, will be executed on Tomcat thread pool, which, in the case of setting spring.threads.virtual.enabled to true, is by default a Tomcat-provided VirtualThreadExecutor and it does not offer graceful shutdown of its threads.

To provide graceful shutdown in this case, it is necessary to configure Tomcat Web server and implement WebServerFactoryCustomizer to override Spring Boot-provided TomcatWebServerFactoryCustomizer:

@Configuration
public class TomcatConfig implements WebServerFactoryCustomizer<ConfigurableTomcatWebServerFactory>, Ordered {

    private static final int TomcatWebServerFactoryCustomizer_ORDER = 0;

    @Override
    public int getOrder() {
        return TomcatWebServerFactoryCustomizer_ORDER + 1;
    }

    @Override
    public void customize(ConfigurableTomcatWebServerFactory factory) {
        factory.addProtocolHandlerCustomizers(
            (protocolHandler) -> {
            protocolHandler.setExecutor(getVirtualThreadPool());
        });
    }

    @Bean(destroyMethod = "close")
    public ExecutorService getVirtualThreadPool() {
        return Executors.newVirtualThreadPerTaskExecutor();
    }

}

Executors.newVirtualThreadPerTaskExecutor() will create ThreadPerTaskExecutor with a virtual thread factory, it is a non-pooling Executor which nevertheless implements graceful shutdown.

Application Context close()

Note that graceful shutdown of virtual threads in all cases depends on invocation of Spring Application Context close() method and subsequent invocation of @PreDestroy methods, destroyMethods, explicit or implicit, and similar. If the application is closed by taskkill /F <PID> in Windows, kill -9 <PID> in Unix or similar means that do not initiate graceful shutdown of Spring Application Context, then none of the solutions, described above, will work. There is a complex How to shutdown a Spring Boot Application in a correct way? thread that discusses the details of graceful shutdown of Spring Boot Application.

Spring Application Context is closed gracefully if the Spring Boot-provided actuator/shutdown endpoint is invoked. Alternatively, you could implement your own endpoint which calls Application Context close() method. For example:

@Controller
public class MyShutdownController {

    @Autowired
    private ConfigurableApplicationContext  ctxt;

    @RequestMapping("shutdown")
    @ResponseBody
    public String shutdown() {
        Thread.ofPlatform().start( () -> {
            ctxt.close();
        });
        return "Closing in progress...";
    }

}

It might be important in this case to call Application Context close() method on a new, dedicated platform thread - exactly what Spring's ShutdownEndpoint is doing. Otherwise, if this endpoint itself is executed on a virtual thread, the Executor will wait forever for a termination of this exact thread.

Answer 2

Update: Changed link to latest doc.

From Spring docs

One side effect of virtual threads is that they are daemon threads. A JVM will exit if all of its threads are daemon threads. This behavior can be a problem when you rely on @Scheduled beans, for example, to keep your application alive. If you use virtual threads, the scheduler thread is a virtual thread and therefore a daemon thread and won’t keep the JVM alive. This not only affects scheduling and can be the case with other technologies too. To keep the JVM running in all cases, it is recommended to set the property spring.main.keep-alive to true. This ensures that the JVM is kept alive, even if all threads are virtual threads.