CODEWITHSUNDEEP
neural-networkconvolutionkeras
B_Miner
B_Miner

Reputation: 1820

Keras - 1D Convolution How it works

From this example: https://github.com/fchollet/keras/blob/master/examples/imdb_cnn.py

comes this snippet below. The embedding layer outputs a 400 x 50 matrix for each example in a batch. My question is how does the 1D convolution work? How does it work across the 400 x 50 matrix?

# we start off with an efficient embedding layer which maps
# our vocab indices into embedding_dims dimensions
model.add(Embedding(max_features,
                    embedding_dims,
                    input_length=maxlen,
                    dropout=0.2))

# we add a Convolution1D, which will learn nb_filter
# word group filters of size filter_length:
model.add(Convolution1D(nb_filter=nb_filter,
                        filter_length=filter_length,
                        border_mode='valid',
                        activation='relu',
                        subsample_length=1))

Upvotes: 11

Views: 10072

Answers (2)

berthié gouin
berthié gouin

Reputation: 11

Coming from a background of signal processing it also took me while to understand the concept of it and it seems to be the case of many people in the community.

Pyan gave a very good explanation. As it is often explained with words in many forums, I made a little animation the I hope will help.

See below the input tensor, the filter (or weight) and the outputed tensor. You can also see the size of the output tensor as a function of the number of filters used (represented with different colours).

Visual Representation of the 1D Convolultion (Simplified)

Note that to perform the scalar multiplication between the input and the filter, the filter should be transposed. There are also different implementations (Karas, Tensorflow, Pytorch...), but I think this animation can give a good representation of what is happening.

Hope it can help someone.

Upvotes: 1

pyan
pyan

Reputation: 3707

In convolutional neural networks (CNNs), 1D and 2D filters are not really 1 and 2 dimensional. It is a convention for description.

In your example, each 1D filter is actually a Lx50 filter, where L is a parameter of filter length. The convolution is only performed in one dimension. That may be why it is called 1D. So, with proper padding, each 1D filter convolution gives a 400x1 vector. The Convolution1D layer will eventually output a matrix of 400*nb_filter.

Upvotes: 13

Related Questions