Basic C++ Multithreading

Question

I'm in the design phase of development and am considering using multi-threading in C++ to implement some functionality. I'm familiar with the basics of multi-threading but wanted to get others take on my idea. I haven't chosen a multi-threading library yet (leaning towards Boost) but my question is probably independent of the library chosen.

Basically I would have a class (let's call it CommandGenerator) that executes in a while loop (until terminated) and checks a message queue of commands that is populated by another piece of software. Every time CommandGenerator gets a message off the queue, I'd like it to spawn a thread that executes in the background and works with the data just pulled off the queue. Meanwhile I want CommandGenerator to continue to run and come around the while loop again and pull any new messages and again spawn more threads. Is this conceptually possible? Can I keep spawning threads and just let them run in the background until they complete while the code continues to loop and check the queue? CommandGenerator will not need to have control over the threads. They would be able to execute independently once created and are guaranteed to terminate but may take up to a minute to finish executing (they wait a certain amount of time specified in the message pulled off the queue before executing).

Any input is appreciated.

Answer 1

Your concept is feasible and quite close to some regular multi-threading patterns. One pattern is to use have a shared thread safe queue. You have a producer, in this case your CommandGenerator that pushes work onto the queue. You then have consumer threads that sit waiting for something to be put onto the queue. When an item is pushed onto the queue, one (and only one) of the threads is able to take the item off and process it.

One way to implement this is using std::thread and std::mutex.

Depending on the compiler you're using, you should seriously look at std::async. This can remove the burden of needing to know what threads will be created and when.

You will need to make sure you know what happens when you need to stop processing. For example, when your CommandGenerator is stopped, should it wait for processing threads to stop? How are command processing threads stopped? What happens to items still in the queue? These are design decisions that you will need to think about (if you haven't already).

Answer 2

What you want to do is called a "producer-consumer" pattern.

I would strongly advise against creating a new thread for each message received: if you did that and you get too many messages at once you can clog up the machine.

Instead, have a fixed number of consumer threads that read the message queue, and let them handle one message at a time. If too many messages come at a single time they will be stored in the queue, waiting to be processed.

Since you have a delay between fetching the message and actually processing it, again the solution IMHO is not to spawn one thread per message but just to increase the number of your consumer threads. This way you can keep resource usage under control. How many threads exactly you will need is totally dependent on your application, you will have to find this out yourself.

As for the implementation, if you use C++11 you just need std::thread / std::mutex and std::condition_variable. If you use C++03 then boost has the equivalent classes.

Answer 3

Yes, your concept is quite feasible. However, nominally you'd have an upper bound on how many threads that can be spawned. The bound can be large though.