CODEWITHSUNDEEP
c++multithreadinglock-free
NrubDub
NrubDub

Reputation: 11

Single producer, multiple consumer architecture

I am faced with parallelizing a simple poll-process loop that looks like this:

while(!done)
    buffer = poll(...)
    foreach(item i in buffer)
        process(i)

the problem follows a single producer/multiple consumers pattern except the items produced have to be consumed by all consumers.

Considering the implementation will have to be multi-threaded C++ code, what kind of data structures should I be using?

Thanks for your advice!

Upvotes: 1

Views: 1351

Answers (2)

zmbq
zmbq

Reputation: 39039

If you know the number of consumers in advance, you can have a queue per consumer.

If consumers are added and removed dynamically, you have to decide what to do with messages generated before a consumer was created. If all consumers must handle all messages - use a vector to keep all messages, and have each consumer keep just an index for the last processed message in the server.

Don't forget to synchronize your access to the vector, since it can be reallocated when the producer adds an item

Upvotes: 1

ichramm
ichramm

Reputation: 6642

Si I guess the best option is to create a message queue for each consumer.

This means all the consumers have to register with the producer, and then, the producer should queue the message to each message queue.

A good implementation option should be to create a proxy object between the producer and the consumer, the proxy object is an hybrid object, which means it consumes all the messages from the producer but also forwards them to all of the registered consumers.

Upvotes: 0

Related Questions