Using Virtual Threads (Project Loom) with Spring WebFlux/Reactor/Reactive libraries

Question

Java Virtual Threads

In Java 19 were introduced Virtual Threads JEP-425 as a preview feature.

After some investigation of concept of Java Virtual Threads (Project Loom), called sometimes lightweight threads (or sometimes fibers or green threads), I'm pretty interested in potential usage of them with reactive libraries, like, for example, with Spring WebFlux based on Project Reactor (reactive streams implementation) and Netty, for making blocking calls efficiently.

Most JVM implementations today implement Java threads as thin direct wrappers around operating system threads, called sometimes heavyweight, OS-managed threads platform threads.

While a platform thread can only execute a single thread at a time, the virtual threads have the ability to switch to executing a different virtual thread when the currently executed virtual thread makes a blocking call (e.g. network, file system, database call).

How do we deal with blocking calls in Reactor?

So, when dealing with blocking calls in Reactor we use the following construct:

Mono.fromCallable(() -> {
     return blockingOperation();
}).subscribeOn(Schedulers.boundedElastic());

In subcribeOn() we provide a Scheduler that creates a dedicated thread for executing that blocking operation. However, it means that the thread will eventually be blocked, so, as we are still on the old-fashioned threading model, we'll actually block the platform thread, which is still not really efficient way of dealing with CPU resources.

Here is the question:

So, the question is, could we use the virtual threads with reactive frameworks directly for making blocking calls like this, using, for example, Executors.newVirtualThreadPerTaskExecutor() :

Creates an Executor that starts a new virtual Thread for each task. The number of threads created by the Executor is unbounded.

Mono.fromCallable(() -> {
    return blockingOperation();
}).subscribeOn(Schedulers.fromExecutorService(Executors.newVirtualThreadPerTaskExecutor()));

Would it work out of the box? Will we actually get the benefits from this approach, in terms of dealing with our CPU-resources more efficiently and increase performance of our application? Does it mean that we can easily integrate reactive library with any blocking library/framework, for example, Spring Data JPA (which is based on JDBC) and millions of others and magically turn them to non-blocking?

Answer 1

Blocking in Reactive code

You can also block in Reactive code, it's just normally not a good idea.

Without virtual threads, doing a blocking operation would also block the platform thread so you'd essentially be wasting the platform thread if you block in reactive code.

with Project Loom

If you are using Executors.newVirtualThreadPerTaskExecutor(), this is not a problem any more (at least in most cases, there are some exceptions for e.g. native-code or when blocking in synchronized blocks - in these cases, the virtual thread, is 'pinned' to a platform thread).

The problems

An issue with that is that you are breaking the paradigm. While your project is reactive, some of your code isn't and you are ending up with a codebase that uses reactive code for some parts and doesn't for other parts. However, that might be fine temporarily if you are in the process of migrating an existing Reactive project to synchronous code using virtual threads and you are planning to remove the reactive framework from your project (you could do so incrementally).

However, note that Virtual Threads are (at the time of writing this) still a preview feature so there may be breaking changes to it. So, you might not want to migrate to virtual threads yet and wait until virtual threads go out of preview as switching back might be pretty difficult. Virtual Threads were finalized in Java 21.

In any way, don't start a project using a Reactive framework but blocking inside Reactive code just because you are using Loom's virtual threads. Instead, either choose the Reactive programming style or use virtual threads and write it synchronously. If you have specific work that should be done in a platform thread (e.g. CPU-intensive work), nothing stops you from creating a platform thread for that.

After all, the entire purpose of Reactive programming is not to block threads. If you want to block (virtual) threads, there's no point in using reactive frameworks (at least in my opinion).

Migrating from Reactive to virtual threads

How you are migrating from reactive code to virtual threads depends on you and your project. If your project is modularized properly, then it would probably be a good idea to choose the modularized approach and migrate one module after the other.

If you have a codebase using a reactive framework and want to switch to virtual threads, you can start by configuring the reactive framework to use virtual threads itself using Executors.newVirtualThreadPerTaskExecutor() or similar. This is not necessary for a top-down approach as long as the blocking code runs in virtual threads.

Top-down

You can try to migrate it top-down - you make sure that the calling code is executed in virtual threads and rewrite that step by step to block. For that, you could (since you are in a virtual thread) run some reactive operation and block for the final result of the operation. You can then later continue rewriting the reactive code that has been called by the reactive operation when no other reactive code is using this operation (all code calling the operation has been rewritten to virtual threads).

bottom-up

Reactive frameworks typically provide a way to run non-reactive code using another thread or similar. You can configure this to use virtual threads and then start rewriting the code bottom-up.

In order to do this, you would start rewriting reactive code that doesn't depend on other reactive code and change the calling code to treat it like non-reactive code (while making sure it runs in virtual threads).

mixed

You could also start in the middle. For that, you really need to make sure that your reactive framework runs your code in virtual threads.

Then, you would block for called reactive code while the code calling your code treats your rewritten code as non-reactive code.

However, you should note that this will lead to the aforementioned problems with breaking the paradigms. You will probably have half-reactive half-blocking code which might be annoying to deal with.

modularized

If you do a big rewrite of your codebase, you might end up in a situation where a part is rewritten and other parts aren't.

If your application has clearly separated modules (could be microservices but monolithic applications can also be modularized), you can start by rewriting one module after another. If you are using microservices, you should be able to rewrite one microservice at a time without impacting other microservices.