CODEWITHSUNDEEP
c++multithreadingsocketsboost-asio
waterlooalex
waterlooalex

Reputation: 13912

I have a server listening on sockets, whats a good approach to service CPU-bound requests with multiple threads?

I've got an application, written in C++, that uses boost::asio. It listens for requests on a socket, and for each request does some CPU-bound work (e.g. no disk or network i/o), and then responds with a response.

This application will run on a multi-core system, so I plan to have (at least) 1 thread per core, to process requests in parallel.

Whats the best approach here? Things to think about:

Currently the server is single threaded:

Update:

More specifically: what mechanism should I use to ensure that if the server is busy that incoming requests get queued up? What mechanism should I use to distribute incoming requests among the N threads (1 per core)?

Upvotes: 2

Views: 858

Answers (3)

Modicom
Modicom

Reputation: 299

If you are using the basic_socket_acceptor's overloaded constructor to bind and listen to a given endpoint, it uses SOMAXCONN as the backlog of pending connections in the call to listen(). I think (not very sure) that this maps to 250 in Windows. So the network service provider will (silently) accept client connections up to this limit and queue them for your application to process. Your next accept call will pop a connection from this queue.

Upvotes: 1

Newton Falls
Newton Falls

Reputation: 2198

I don't see that there is much to consider that you haven't already covered.

If it is truly CPU-bound then adding threads beyond the number of cores doesn't help you much except if you are going to have a lot of requests. In that case the listen queue may or may not meet your needs and it might be better to have some threads to accept the connections and queue them up yourself. Checkout the listen backlog values for your system and experiment a bit with the number of threads.

UPDATE:

listen() has a second parameter that is your requested OS/TCP queue depth. You can set it up to the OS limit. Beyond that you need to play with the system knobs. On my current system it is 128 so it is not huge but not trivial either. Check your system and consider whether you realistically need something larger than the default.

Beyond that there are several directions you can go. Consider KISS - no complexity before it is actually needed. Start off with something simple like just have a thread to accept connection (up to some limit) and plop them in a queue. Worker threads pick them up, process, write result, and close socket.

At the current pace of my distro's Boost updates (and my lack of will to compile it myself) it will be 2012 before I play with ASIO - so I can't help with that.

Upvotes: 2

anio
anio

Reputation: 9171

ACE http://www.cs.wustl.edu/~schmidt/ACE/book1/

It has everything you need. Thread management and queues and as an added bonus a portable way of writing Socket Servers.

Upvotes: 1

Related Questions