Dropwizard: MaxQueuedRequests configuration

Question

I want to restrict my dropwizard incoming request to 3. So if the server is servicing 3 requests, I want to send a 503: "server capacity reached" response Code.

I know Dropwizard has configuration for max queued requests:

MaxQueuedRequests

This defaults to 1024. However, I dont know how to play with this parameter. When I make this 3, my server wont start up.

If anyone can help provide more info on what is the preferrable design pattern to use to achieve this?

One way I am thinking of doing: is to have a Global (or static) variable to keep a counter. And for every requests, I will increment the counter. when counter > 3, I respond with Code 503.

Want to hear more thoughts on this.

edit:

So I was able to make progress following @pandaadb answer. However, I hit another roadblock :( This is what my configuration looks like:

server:
   minThreads: 2
   maxQueuedRequests: 3
   type: simple
   applicationContextPath: /
   adminContextPath: /admin
   connector:
       type: http
       port: 9999

With this, i keep getting the following exception:

WARN  [2016-06-23 08:38:52,354] io.dropwizard.setup.AdminEnvironment: 
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!    THIS APPLICATION HAS NO HEALTHCHECKS. THIS MEANS YOU WILL NEVER KNOW      !
!     IF IT DIES IN PRODUCTION, WHICH MEANS YOU WILL NEVER KNOW IF YOU'RE      !
!    LETTING YOUR USERS DOWN. YOU SHOULD ADD A HEALTHCHECK FOR EACH OF YOUR    !
!         APPLICATION'S DEPENDENCIES WHICH FULLY (BUT LIGHTLY) TESTS IT.       !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
INFO  [2016-06-23 08:38:52,358] org.eclipse.jetty.server.handler.ContextHandler: Started i.d.j.MutableServletContextHandler@9d200de{/admin,null,AVAILABLE}
WARN  [2016-06-23 08:38:52,363] org.eclipse.jetty.util.thread.QueuedThreadPool: dw{STARTED,2<=3<=1024,i=0,q=3} rejected org.eclipse.jetty.util.thread.NonBlockingThread@109d724c
WARN  [2016-06-23 08:38:52,367] org.eclipse.jetty.util.component.AbstractLifeCycle: FAILED org.eclipse.jetty.server.ServerConnector$ServerConnectorManager@289ba785: java.util.concurrent.RejectedExecutionException: org.eclipse.jetty.util.thread.NonBlockingThread@109d724c
! java.util.concurrent.RejectedExecutionException: org.eclipse.jetty.util.thread.NonBlockingThread@109d724c
! at org.eclipse.jetty.util.thread.QueuedThreadPool.execute(QueuedThreadPool.java:362) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.io.SelectorManager.execute(SelectorManager.java:160) ~[jetty-io-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.io.SelectorManager.doStart(SelectorManager.java:258) ~[jetty-io-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) [jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106) [jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:256) [jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81) [jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236) [jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.Server.doStart(Server.java:366) [jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at io.dropwizard.cli.ServerCommand.run(ServerCommand.java:43) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:76) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.Cli.run(Cli.java:70) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.Application.run(Application.java:73) [dropwizard-core-0.8.2.jar:0.8.2]
! at net.lacework.eventgen.EventGeneratorMain.main(EventGeneratorMain.java:16) [classes/:na]
! at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
! at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_77]
! at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77]
! at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77]
! at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) [idea_rt.jar:na]
WARN  [2016-06-23 08:38:52,368] org.eclipse.jetty.util.component.AbstractLifeCycle: FAILED query@3e8f7922{HTTP/1.1}{0.0.0.0:9999}: java.util.concurrent.RejectedExecutionException: org.eclipse.jetty.util.thread.NonBlockingThread@109d724c
! java.util.concurrent.RejectedExecutionException: org.eclipse.jetty.util.thread.NonBlockingThread@109d724c
! at org.eclipse.jetty.util.thread.QueuedThreadPool.execute(QueuedThreadPool.java:362) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.io.SelectorManager.execute(SelectorManager.java:160) ~[jetty-io-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.io.SelectorManager.doStart(SelectorManager.java:258) ~[jetty-io-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:256) ~[jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81) ~[jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236) ~[jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.Server.doStart(Server.java:366) [jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at io.dropwizard.cli.ServerCommand.run(ServerCommand.java:43) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:76) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.Cli.run(Cli.java:70) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.Application.run(Application.java:73) [dropwizard-core-0.8.2.jar:0.8.2]
! at net.lacework.eventgen.EventGeneratorMain.main(EventGeneratorMain.java:16) [classes/:na]
! at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
! at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_77]
! at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77]
! at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77]
! at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) [idea_rt.jar:na]
WARN  [2016-06-23 08:38:52,368] org.eclipse.jetty.util.component.AbstractLifeCycle: FAILED org.eclipse.jetty.server.Server@70242f38: java.util.concurrent.RejectedExecutionException: org.eclipse.jetty.util.thread.NonBlockingThread@109d724c
! java.util.concurrent.RejectedExecutionException: org.eclipse.jetty.util.thread.NonBlockingThread@109d724c
! at org.eclipse.jetty.util.thread.QueuedThreadPool.execute(QueuedThreadPool.java:362) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.io.SelectorManager.execute(SelectorManager.java:160) ~[jetty-io-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.io.SelectorManager.doStart(SelectorManager.java:258) ~[jetty-io-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:256) ~[jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81) ~[jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236) ~[jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.Server.doStart(Server.java:366) ~[jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at io.dropwizard.cli.ServerCommand.run(ServerCommand.java:43) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:76) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.Cli.run(Cli.java:70) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.Application.run(Application.java:73) [dropwizard-core-0.8.2.jar:0.8.2]
! at net.lacework.eventgen.EventGeneratorMain.main(EventGeneratorMain.java:16) [classes/:na]
! at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
! at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_77]
! at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77]
! at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77]
! at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) [idea_rt.jar:na]
ERROR [2016-06-23 08:38:52,368] io.dropwizard.cli.ServerCommand: Unable to start server, shutting down
! java.util.concurrent.RejectedExecutionException: org.eclipse.jetty.util.thread.NonBlockingThread@109d724c
! at org.eclipse.jetty.util.thread.QueuedThreadPool.execute(QueuedThreadPool.java:362) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.io.SelectorManager.execute(SelectorManager.java:160) ~[jetty-io-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.io.SelectorManager.doStart(SelectorManager.java:258) ~[jetty-io-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:256) ~[jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81) ~[jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236) ~[jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.server.Server.doStart(Server.java:366) ~[jetty-server-9.2.9.v20150224.jar:9.2.9.v20150224]
! at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.2.9.v20150224.jar:9.2.9.v20150224]
! at io.dropwizard.cli.ServerCommand.run(ServerCommand.java:43) ~[dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:76) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.cli.Cli.run(Cli.java:70) [dropwizard-core-0.8.2.jar:0.8.2]
! at io.dropwizard.Application.run(Application.java:73) [dropwizard-core-0.8.2.jar:0.8.2]

Ideally, I would want to also try doing limiting my server requests to 1. So that it services only 1 request at a time. Also, I dont know the new requests coming in to WAIT. Instead, they server should reject them with some response code.

I am not sure if dropwizard already has this built in.

Answer 1

I think your issue is that you only change one value, not the others. And DW validation does not catch that case.

Setting your MaxQueuedRequests does not allow 3, if you do not change your capacity as well. Look here:

/**
     * Create a bounded {@link BlockingArrayQueue} that grows by the given parameter.
     * 
     * @param capacity
     *            the initial capacity
     * @param growBy
     *            the growth factor
     * @param maxCapacity
     *            the maximum capacity
     */
    public BlockingArrayQueue(int capacity, int growBy, int maxCapacity)
    {
        if (capacity > maxCapacity)
            throw new IllegalArgumentException();
        _elements = new Object[capacity];
        _growCapacity = growBy;
        _maxCapacity = maxCapacity;
    }

This is called by the server factory:

protected ThreadPool createThreadPool(MetricRegistry metricRegistry) {
        final BlockingQueue<Runnable> queue = new BlockingArrayQueue<>(minThreads, maxThreads, maxQueuedRequests);
        final InstrumentedQueuedThreadPool threadPool =
                new InstrumentedQueuedThreadPool(metricRegistry, maxThreads, minThreads,
                                                 (int) idleThreadTimeout.toMilliseconds(), queue);
        threadPool.setName("dw");
        return threadPool;
    }

Meaning your minThreads need to be 3 or less for this to work, otherwise your capacity is bigger than maxCapacity, where:

capacity: minThreads maxCapacity: maxQueuedRequests

EDIT: After more info from Op, here's the solution:

server:
  minThreads: 2
  maxQueuedRequests: 3
  rootPath: /api/*
  requestLog:
    appenders: []
  applicationConnectors:
  - type: http
    port: 9085
    acceptorThreads: 3
    selectorThreads: 3

your error is not really an error - i believe it is a bug in Jetty (or rather a misconfiguration that is not validated on startup).

The issue is that jetty starts a few acceptorthreads/selectorthreads. These are count() > 3. However the threads are started by the server lifecycle, so jetty bombs them into its own threadpool. This is no problem since they go away once executed (you can test this by setting a breakpoint and slowly stepping through the startup - your server will start just fine).

The solution is to limit the acceptor/selector threads (please do some experiments/research to see what the things do and configure them correctly). So by limiting those threads, the startup does not overload your jetty, and everything starts fine.

(I did not do any experiments regarding performance/stability with this rather odd setup).

FWIW - the way I would solve this issue:

Implement a request Filter that counts incoming/outgoing requests. Once a request comes in and the counter is > X, reject the request immidiately with a response indicating high load and later retry (so your client knows what to do). The above config will likely result in 500's thrown from jetty which will make your clients think your server is down.

Regards,

Artur