CODEWITHSUNDEEP
javajava-threadsproject-loom
Andrew Parks
Andrew Parks

Reputation: 8087

Minimum concurrency to benefit from JDK19 virtual threads

According to JEP 425,

virtual threads can significantly improve application throughput when the number of concurrent tasks is high (more than a few thousand), and the workload is not CPU-bound

Why are virtual threads not helpful when the thread count is much lower than a few thousand? If I have 50 concurrent I/O-bound tasks, will I not significantly reduce CPU load by using virtual threads to eliminate heavyweight OS thread context switching?

Upvotes: 3

Views: 864

Answers (2)

Borislav Stoilov
Borislav Stoilov

Reputation: 3687

The default number of threads in one of the most popular servers (tomcat) is 200. Any number of threads above the core count will result in context switches, this includes threads that are not owned by your process.

There is overhead even for idle (sleeping) threads. The scheduler has to ping them from time to time to see if any of them needs to be awakened. This means that it is always a good idea to keep the thread count low and as close to the number of cores as possible.

No harm in using VT as mentioned by the other answers you should experiment and see how they behave in your app but don't expect any change for that tiny load

Upvotes: 1

raiks
raiks

Reputation: 1520

In short: in case of 50 concurrent tasks you won't gain much by using virtual threads. They won't make things worse either - you'll just hardly notice any difference. But when the number of native threads is high, things start to change.

How high? A typical application can sustain only a several thousand threads running simultaneously (an empirical observation). Beyond that you'll most likely run out of RAM (thread stacks will take tens of gigabytes of memory), plus constant context switching will slow down your application.

Virtual threads are switched in user space and their stacks are tiny (hundreds of bytes). Thus, your application will likely survive millions of them.

So, unless you have a need to big number of clients simultaneously (by big I mean hundreds and thousands), you won't benefit much from using virtual threads. Those are for monstrous I/O bound workloads. But I suggest try and measure, of course.

Upvotes: 2

Related Questions