CODEWITHSUNDEEP
pythontensorflowkeras
Andrew Cassidy
Andrew Cassidy

Reputation: 2998

Simple Conv1D as first layer in keras

Here is my input

x_train.shape # (12, 7) 12 observations each of length 7
x_train # dtype('int32')

Here's the architecture I'd like to achieve:

enter image description here

I'd like a kernel of size 3 convolved over the sequence. From keras documentation at https://keras.io/layers/convolutional/

"When using this layer as the first layer in a model, provide an input_shape argument (tuple of integers or None, e.g. (10, 128) for sequences of 10 vectors of 128-dimensional vectors, or (None, 128) for variable-length sequences of 128-dimensional vectors."

Honestly I'm having a hard time understanding their logic. Here's my attempt

docs_sequence = Input(shape=(7,), dtype='float32') # Longest document is 7 words
convolution = Conv1D(filters = 1,  # only 1 convolution
                     kernel_size = 3, # tri grams
                     strides = 1,
                     input_shape = (1, 7),
                     padding = 'valid',
                     activation = 'relu')(docs_sequence)
output = Dense(1, activation='sigmoid')(convolution)
cnn_model = Model(inputs = docs_sequence, outputs = [output])
cnn_model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

and I'm consistently getting

ValueError: Input 0 is incompatible with layer conv1d_30: expected ndim=3, found ndim=2

Upvotes: 3

Views: 3174

Answers (1)

Andre Holzner
Andre Holzner

Reputation: 18695

As the error message says, your input is two dimensional while the convolutional layer expects a three dimensional input.

With the following

docs_sequence = Input(shape=(7,1), ...

instead of 

docs_sequence = Input(shape=(7,), ...

Keras accepts the model. Basically this adds a dimension of size one to the input (the three dimensions from the error message include the minibatch dimension which one can think of being prepended to the shape argument above).

cnn_model.summary() then gives:

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         (None, 7, 1)              0
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 5, 1)              4
_________________________________________________________________
dense_1 (Dense)              (None, 5, 1)              2
=================================================================

When preparing the actual input data, you may have to add this dimension of size one to your input data. You may want to use numpy.atleast_2d() or numpy.atleast_3d() for this, possibly combined with taking the transpose or use numpy.expand_dims().

In your case, np.atleast_3d(x_train) gives a shape of (12, 7, 1).

Upvotes: 5

Related Questions