CODEWITHSUNDEEP
tensorflowneural-networkdeep-learningconv-neural-network
Sven E.
Sven E.

Reputation: 13

Difference between parallel and sequential Convolutions in Convolutional Neural Network

I'm trying to implement a Convolutional Neural Networks with tensorflow for classifying text. I already found some implemented models and found especially two implementations:

[I'm not allowed to post more than 2 Links, I will try to provide the sources in the comments]

However they seem to differ fundamentally in their architecture. The first model uses the convolutions parallel with the input data whereas the second model uses the convolutions in a sequential way. I visualized the two models with tensorboard:

First the parallel convolutions. After the convolutions the results are concated and with one fully connected layer the output is created.

parallel use of convolutions

The sequential convolutions seem to be more straight forward, we use the result form the previous layer as the input for the next layer.

sequential use of convolutions

So my question is, since both are used to classify text, where lies the difference between these two implementations and which one is more suitable for classifying text?

Upvotes: 1

Views: 2642

Answers (1)

Thomas Wagenaar
Thomas Wagenaar

Reputation: 6759

This is not necessary 'parallel' vs. 'sequential'. From what i'm seeing is that the 'parallel' implementation is actually just óne convolutional layer, however with different filter sizes.

Basically, if you just had a convolutional layer with 93x3 features it would be:

input                 convoluted                  pooling         
x * y * 1             x' * y' * 9                 x'' * y'' * 9

So basically, each filter undergoes the same pooling and convolutional operation.

But the main difference in your 'sequential' model is that he's using features with different filter sizes. 

input                 convoluted                  pooling          concat (may also flatten
x * y * 1             x1 * y1 * 3                 x1' * y1' * 3    x123' * y123' * 9
                      x2 * y2 * 3                 x2' * y2' * 3
                      x3 * y3 * 3                 x3' * y3' * 3

Which then get concatted into óne again. The only difference between this and the 'sequential' model is that he is explicitly showing the different filter sizes - but the number of features is exactly the same.

Your 'parallel' and 'sequential' are both just as parallel: each of the feature maps gets convoluted & pooled individually.

Upvotes: 3

Related Questions