CODEWITHSUNDEEP
c++design-patterns
eoldre
eoldre

Reputation: 1087

c++ design patterns for chaining together transformations of streams of objects

I'm working on a multithreaded library which monitors network traffic from winpcap and transforms the packets into several different types of data structures for consumption by various applications.

for each type of output, there will be several transformations required, each transformation could be described as taking 0-N objects of type X, and then producing 0-N types of Y which will then be consumed by the next step in the process.

It's important to note that in the transformation of X's to Y's. If we currently only have 5 (as an example) X's, that may or may not be enough to create a Y, or it might be enough to create many Y's, depending on the transformation and the data recieved.

To be consistant, we would obviously like to use a standard pattern for each transformation object. I'm hoping that someone could point out a commonly used pattern for something like this that hopefully relies on std (or boost) libraries.

Additionaly we have been discussing the possibility of using chains of inheritance to link the different layers together.

IE.

class ProcessXtoY: ProcessWtoX
{
    void processData(iterator<X> begin, iterator<X> end)
    {
        /* create Y's, send output to  */
    }
    virtual void processData(iterator<Y> begin, iterator<Y> end) = 0;
}

class ProcessYtoZ: ProcessXtoY
{
    void processData(iterator<Y> begin, iterator<Y> end)
    {
        /* ... */
    }
}

Can anyone suggest some examples of commonly used patterns for this type of project?

Upvotes: 4

Views: 1363

Answers (3)

James
James

Reputation: 1804

I have a few suggestions. You could use a variation of the Decorator Pattern. You can modify this pattern so you chain together different object types. Then if you have different implementations of the same transform it is easy to problematically, or at runtime, swamp it out. There might be a pattern, already named, somewhere that is this variant, but you should be able to derive it from the basics of the Decorator Pattern.

If you want a multi-threaded solution I would recommend chaining together your transformations through a producer/consumer queue (see this). That way you could actually have several different consumers (transforms) work on in parallel and place the completed transforms onto the next producer/consumer in the line. Of course, this really only works when if ordering of your transforms don't matter for the rest of your program, or you have some way to keep track of that and reorder the final objects when they are needed again. Again, you can easily swap out your transforms programatically or at runtime if you have different implementations of them.

If you need something more generic and configurable you could use the Builder Pattern to encapsulate the chaining process and allow for complete runtime configuration of the builder and have finer control of swapping out the chaining process. Of course you would use other patterns within the builder to implement the transformation chains.

Upvotes: 1

Mark B
Mark B

Reputation: 96241

Using inheritance to link the transformations together is not what inheritance should be used for and pretty unflexible in adding new transformations. If you ever need new combinations of transformations (for example W directly to Y).

Instead, have you considered creating transformation class(es) that describe each transformation algorithm and then use std::transform and chain the transformations together?

Upvotes: 2

user3458
user3458

Reputation:

The approach in your sample (I'd call it "iterate" approach) is an obvious strawman - you're pushing an infinite stream of packets, so there is no end() to them.

I think you can go with a pull or a push approaches. For pull, something along Java's hasNext()/next(). Unfortunately, it's hard to branch, and the original data source needs to queue because we don't know when consumers will pick up the pockets.

For push approach you can use register(listener) and listener.process() combination. This one is easily branched and the buffering (in case process() at the network packet layer takes too long) can be done for you by the system, or you can introduce explicit queues at any level.

So overall, I'd recommend event listeners here.

Upvotes: 1

Related Questions