Ask about how jetty works embedded in maven

Question

I been using jetty in maven project.

It is working with the flow below:

I overwrite QueueThreadPool of Server class of jetty:

QueuedThreadPool threadPool = new QueuedThreadPool(StaticConfig.JETTY_MAX_THREADS, 
StaticConfig.JETTY_MIN_THREADS, StaticConfig.JETTY_IDE_TIMEOUT, 
new BlockingArrayQueue<Runnable>(maxCapacity));

with JETTY_MAX_THREADS=50, JETTY_MIN_THREADS=5

but I have 5 thread created in monitor when start server:

xxx_thread-14
xxx_thread-15
xxx_thread-16
xxx_thread-17-acceptor-0@3d01ec3d-ServerConnector@11758f2a{HTTP/1.1, (http/1.1)}{0.0.0.0:90}
xxx_thread-18

threads 14,15,16,18 is threads in Thread Pool (img diagram)? what is Connection Queue?

How create mutils acceptor threads?

please help me handle request to server with diagram in first image with number thread created in config.

Thank you very much.

Answer 1

Dear @Joakim Erdfelt,

I apply above config but I have problem with Memory of server (at current set 3GB), total request per time process about 2500!!

I don't understand why memory as full? because when thread closed when HeapByteBuffer will reset free!!

And I have open 8 acceptor and 8 selector, when class Server of package jetty I see comment selector will use when acceptor full, but I monitor threads I had seen selector used, acceptor as free :(

please suggest me, Thanks.

Answer 2

@Joakim Erdfelt, Thank you very much for reply!!

My server live in diagram microservices not as web server, number of connections ould not be reached to 30,000 that might be Google ^.^.

I just want the task sequence to go out of the webserver in diagram, because I received response TIMEOUT from server of partner if not task sequence.

Time for response about 10s so not have threads busy in min thread.

I been using:

<jetty.version>9.4.30.v20200611</jetty.version>

I think diagram above is core of jetty because I still find it allows to do that, But I have solution advance diagram with config my server:

boolean checkOOM = threadPool.getThreadPoolBudget().check(StaticConfig.JETTY_MAX_THREADS);
LOGGER.info("check maxThread allow with memory : "+checkOOM);
if(!checkOOM) {
   LOGGER.info("please increase merory server or decrease maxThread.");
   System.exit(1);
}
//depend info system
//get core processor of server
int coreProcessor = Runtime.getRuntime().availableProcessors();
LOGGER.info("coreProcessor : "+coreProcessor);

Server server = new Server(threadPool);
int port = 0;

try {
   port = Integer.parseInt(System.getProperty("server.port"));
} catch (NumberFormatException ex) {
  LOGGER.error("Property server.port not found. " + ex.getMessage(), ex);
  System.exit(1);
}
// if acceptor 0 then the selector threads are used to accept connections
// selector backup for acceptor
// only init with limit system
try (ServerConnector httpConnector = new ServerConnector(server, coreProcessor, coreProcessor)) {
    httpConnector.setAcceptQueueSize(StaticConfig.JETTY_MAX_QUEUED);
    LOGGER.info("acceptors : "+httpConnector.getAcceptors());
    LOGGER.info("queueSize : "+httpConnector.getAcceptQueueSize());
    httpConnector.setPort(port);
    server.setConnectors(new Connector[]{httpConnector});
} catch (Exception e) {
    LOGGER.error(e.getMessage(), e);
    System.exit(1);
}
server.start();
server.join(); << init thread pool.

Above code init with config allow in server running, I will have coreProcessor number acceptors and coreProcessor number selectors backup for acceptors.

I used ServerConnector similar to your suggestion.

I try with default QueueThreadPool (init constructor Server not input param threadPool in above code), I haven log print as 4 threads init:

int cntThread = server.getThreadPool().getThreads();
LOGGER.info("idleThread server : "+cntThread);

I am just wondering the role of thread pool in the diagram works correctly!

ps: you have info new diagram of jetty current version?

Thank you very much.

Answer 3

This is a case of premature optimization and not understanding how Jetty, nor your own web application actually works.

Advice, don't prematurely configure the QueuedThreadPool.

Leave it at default, test, and test again, and then do load testing, then do load testing again but in a different way. Gather information the whole time on how the default QueuedThreadPool is behaving.

Then, and only then, you should tune QueuedThreadPool to fit your needs.

But don't ever stop monitoring your server and web application, as you'll be adjusting your configuration often. (especially so if you configure the QueuedThreadPool to be smaller than default)

Important: If you need to limit the number of requests or connection, or control the number of or rate of requests or connections, attempting to do so at the ThreadPool level will never work.

That diagram is wrong (or at least 10 years out of date).

Since Jetty 7, there's no connection queue, there's no separation of Acceptors and Request threads.

The Thread Pool (which is really just a java.util.concurrent.Executor) is used for ALL thread demands on the Jetty server. This could be accepting a connection, handling the nio managed selector, processing the individual nio selectors, processing read events from the network, processing the initial dispatch of a request, processing the write events from the web application to the network, handling async processing from Servlet 3.0, QoSFilter, DoSFilter, handling async I/O from Servlet 3.1, upgraded connections, websocket, HttpSession management, Hot Deployment processing, bytecode scanning, etc.

With a limit of 50 max threads:

• You'll never have a need for another acceptor.
• You'll starve the clients and cause massive issues.
• A typical web page that a browser loads will use between 8 and 12 connections, which means on this 50 max threads configuration will be able to process between 4 and 6 clients concurrently.

The types of applications that need another acceptor are ones that handle massive amounts of new connections (for Eclipse Jetty, you typically need another acceptor when you cross the threshold of 30,000 new connections per second).

The number of acceptors that Jetty uses are configured at the ServerConnector level, not the ThreadPool level.

Look at the ServerConnector constructors and pick the constructor best suited for your environment.