CODEWITHSUNDEEP
javamultithreadingjetty
Andrew
Andrew

Reputation: 419

Jetty, where are my five threads?

I'm using spring+jetty. I'm configuring jetty:

@Bean
public JettyEmbeddedServletContainerFactory jettyEmbeddedServletContainerFactory() {
    final JettyEmbeddedServletContainerFactory factory = new JettyEmbeddedServletContainerFactory(port);
    factory.addServerCustomizers((Server server) -> {
        final QueuedThreadPool threadPool = server.getBean(QueuedThreadPool.class);
        threadPool.setMinThreads(minThreads);
        threadPool.setMaxThreads(maxThreads);
        threadPool.setIdleTimeout(1000);
    });
    return factory;
}

It works, but strange.

  1. Set minThreads=1, maxThreads=5, and it doesn't processing connections.
  2. Set minThreads=1, maxThreads=6, and it processing only one connection
  3. Set minThreads=1, maxThreads=7, and it processing only two connections. When I say processing, I mean, that it accepts connections, but do nothing. Doesn't reply, doesn't abort them. (I think they are in a queue)

So, where are my 5 threads?

Upvotes: 4

Views: 3420

Answers (1)

Joakim Erdfelt
Joakim Erdfelt

Reputation: 49555

You are not accounting for how each of your connectors allocates selectors and acceptors into the same threadpool.

You are also not taking into account the system/hardware effects into the thread pool (how many cpu cores do you have? yes it matters).

You are also defining a rediculously tiny threadpool. Do you plan on only serving 1 http connection in a row? on 1 connector, using HTTP/1.0, with keep-alive, and no compression, with perfect network conditions that never fail or timeout?

Are you 100% sure that your user-agents (clients) will follow those rules?

Hint: the web page you are looking at right now, on stackoverflow, using a modern web browser, would have used from 9 to 18 active threads from the thread pool, grown the thread pool maximum to about 35, and would have been done with them all in 399ms.

Consider that any of the following future decisions would increase your thread pool pressures.

  • Use jetty-proxy: AsyncProxyServlet, AsyncMiddlemanServlet
  • Use WebSockets on the server side.
  • Use WebSocket clients
  • Use Http Clients
  • Enable HTTP/2

Your settings should have thrown an IllegalStateException indicating your very low configuration.

java.lang.IllegalStateException: Insufficient threads: max=8 < needed(acceptors=1 + selectors=8 + request=1)

Since you said you are using Spring, consider following the open issue about this at

https://github.com/spring-projects/spring-boot/issues/8917

Here's some general (large hand waving) advice about tuning for maximum threads ...

  • number of active, in flight, connections is your baseline, start with that as your maximum thread count and move up from there.
  • HTTP/1.x uses less threads then HTTP/2
  • small web resource sizes lean towards lower maximum threads
  • large or long duration web resources lean towards larger maximum threads
  • use of Servlet 3.1+ async I/O means less maximum threads (only uses threads if there is actual read and/or write needed)
  • use of Servlet 3.0 (or older) blocking I/O means more maximum threads (must dedicate thread to each connection read/write)
  • 200 to 500 maximum threads for an average web site, with average load, and average web resource sizes, and average web resource timing, using HTTP/1.x
  • for a heavy web site, using many features (proxy, websockets, httpclient, http/2) a maximum threads configuration of 3,000 to 5,000 is typical.
  • for extreme web sites, dedicate as many CPU cores as you can, use Servlet 3.1 Async I/O exclusively, crank your threads maximum to 9,000 on up to 20,000 (depending on the load profile you have)

The smallest web site we know of using Jetty uses 19MB of memory, and has a configured 8 maximum threads in its thread pool. (its a weather station in a remote part of Norway)

The largest single machine web site we know of using Jetty uses 189GB of memory, on a 30 core machine, with 30,000 thread maximum configured, with 8 network interfaces, HTTP/1.x and HTTP/2 enabled on each, doing SSL + Gzip + WebSockets, with an average of 280,000 active connections at any one moment (peaks at just shy of 800,000 active connections at certain times of day, every day)

Upvotes: 5

Related Questions