CODEWITHSUNDEEP
multithreadingthreadpoolgrpc
Šimon Tóth
Šimon Tóth

Reputation: 36423

Tuning gRPC thread pool

I'm dealing with a legacy synchronous server that has operations running for upto a minute and exposes 3 ports to overcome this problem. There is "light-requests" port, "heavy-but-important" requests port and "heavy" port.

They all expose the same service, but since they run on separate ports, they end up with dedicated thread pools.

Now this approach is running into a problem with load balancing, as Envoy can't handle a single services exposing the same proto on 3 different ports.

I'm trying to come up with a single threadpool configuration that would work (probably an extremely overprovisioned one), but I can't find any documentation on what the threadpool settings actually do.

NUM_CQS     
Number of completion queues.

MIN_POLLERS     
Minimum number of polling threads.

MAX_POLLERS     
Maximum number of polling threads.

CQ_TIMEOUT_MSEC     
Completion queue timeout in milliseconds.

Upvotes: 1

Views: 6772

Answers (1)

Mark D. Roth
Mark D. Roth

Reputation: 839

Is there some reason why you need the requests split into three different thread pools? By default, there is no limit to the number of request handler threads. The sync server will spawn a new thread for each request, so the number of threads will be determined by the number of concurrent requests -- the only real limit is what your server machine can handle. (If you actually want to bound the number of threads, I think you can do so via ResourceQuota::SetMaxThreads(), although that's a global limit, not one per class of requests.)

Note that the request handler threads are independent from the number of polling threads set via MIN_POLLERS and MAX_POLLERS, so those settings probably aren't relevant here.

UPDATE: Actually, I just learned that my description above, while correct in a practical sense, got some of the internal details wrong. There is actually just one thread pool for both polling and request handlers. When a request comes in, an existing polling thread basically becomes a request handler thread, and when the request handler completes, that thread is available to become a polling thread again. The MIN_POLLERS and MAX_POLLERS options allow tuning the number of threads that are used for polling: when a polling thread becomes a request handler thread, if there are not enough polling threads remaining, a new one will be spawned, and when a request handler finishes, if there are too many polling threads, the thread will terminate. But none of this affects the number of threads used for request handlers -- that is still unbounded by default.

Upvotes: 1

Related Questions