CODEWITHSUNDEEP
multithreadingperformancehttpservertbb
Mary Chang
Mary Chang

Reputation: 955

Is sharing TBB's thread pool with a HTTP server a good idea?

I know this is a weird question but hear me out. I'm working on a high throughput, compute heavy HTTP backend server in C++. It is quite straight forward:

  1. Spins up a HTTP server
  2. Receive some request
  3. Do a lot of math
    • This step is parallelized using TBB
  4. Send the result back (takes about 20ms)
  5. There's no limit on how soon the response have to get out. But the lower the worst case the better it is.

Now my bottleneck is the server part of uses a different thread pool than TBB. Thus when TBB is busy doing math. The server may suddenly get tens of new requests, then the thread from the server side get scheduled, and cause a lot of cache miss and branch prediction failures.

A solution I came up is to share TBB's thread pool with the server. Then no request will be registered while TBB is busy and processed immediately after TBB is free.

Is this a good idea? Or could it have potential problems?

Upvotes: 0

Views: 94

Answers (1)

rustyx
rustyx

Reputation: 85341

This is difficult to answer without knowing what that other thread pool is doing. If it handles file or network I/O then combining it with a CPU-intensive pool can be a pessimization since I/O does not consume CPU.

Normally there should be a small pool or maybe even a single thread handling the accept loop and async I/O, handing new requests off to the worker pool for processing and sending the results back to the network.

Try to avoid mixing CPU-intensive work with I/O work, as it makes resource utilization difficult to manage. Having said that, sometimes it's just easier and it's never good to run at 100% CPU anyway. So yes, you should try having just one pool. But measure the performance before/after the change.

Upvotes: 1

Related Questions