How to use 'input_shape' in Conv1D() function?

Question

I have one million rows of data with six features and three classes.

6.442  6.338  7.027  8.789 10.009 12.566  A
6.338  7.027  5.338 10.009  8.122 11.217  A
7.027  5.338  5.335  8.122  5.537  6.408  B
5.338  5.335  5.659  5.537  5.241  7.043  B
5.659  6.954  5.954  8.470  9.266  9.334  C
6.954  5.954  6.117  9.266  9.243 12.200  C
5.954  6.117  6.180  9.243  8.688 11.842  A
6.117  6.180  5.393  8.688  5.073  7.722  A
... ... ... ... ... ... ... ... ... ... ...

I want to feed this dataset into a CNN.

So, I wrote the following Keras code:

model = Sequential()
model.add(Conv1D(filters=n_hidden_1, kernel_size=3, activation='sigmoid', 
        input_shape=(1, num_features)))
model.add(Conv1D(filters=n_hidden_2, kernel_size=3, activation='sigmoid'))
model.add(Dropout(0.5))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(num_classes, activation='softmax'))

This code is giving me the following error:

ValueError: Negative dimension size caused by subtracting 3 from 1 for 
'{{node conv1d/conv1d}} 
= Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1]
, explicit_paddings=[], padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true]
(conv1d/conv1d/ExpandDims, conv1d/conv1d/ExpandDims_1)' with input shapes
: [?,1,1,6], [1,3,6,64].

Edit: Then, I modified the model as follows:

model = Sequential()
model.add(Conv1D(filters=n_hidden_1, kernel_size=3, activation='sigmoid', 
    input_shape=(n_hidden_1, num_features, 1)))
model.add(Conv1D(filters=n_hidden_2, kernel_size=3, activation='sigmoid'))
model.add(Dropout(0.5))
model.add(MaxPooling1D(3))
model.add(Flatten())
model.add(Dense(num_classes, activation='softmax'))

Now I am getting the following error message:

ValueError: Input 0 of layer max_pooling1d is incompatible with the 
layer: expected ndim=3, found ndim=4. Full shape received: 
    (None, 64, 2, 64)

What am I writing wrong and why is it wrong?

Answer 1

Conv1D and MaxPool1D expect input shape like (n_batches, n_steps, n_features). So, input shape should be like input_shape=(n_steps, n_features). And if you want to consider 6 as steps, then it could be like input_shape=(6,1).

1. For adding last dimension try this:

train_X = np.expand_dims(train_x, axis=-1)
validate_x = np.expand_dims(validate_x, axis=-1)

2. Since each convolution layer, with default padding valid, subtract n_steps by 2, then your second dimension shape change is like:

• input -> 6
• After first Conv1D -> 4
• After second Conv1D -> 2

And you can not apply a MaxPool1D by 3 pool size. Either you can change pool size to 2, or add padding="same" to one of your convolution layers:

model = tf.keras.Sequential()
model.add(Conv1D(filters=64, kernel_size=3, activation='sigmoid', input_shape=(6, 1)))
model.add(Conv1D(filters=64, kernel_size=3, activation='sigmoid')) 
model.add(Dropout(0.5))
model.add(MaxPooling1D(2)) # change to 2 or add `padding="same"` to the conv layers
model.add(Flatten())
model.add(Dense(3, activation='softmax'))

model.summary()

Summary:

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv1d_2 (Conv1D)            (None, 4, 64)             256       
_________________________________________________________________
conv1d_3 (Conv1D)            (None, 2, 64)             12352     
_________________________________________________________________
dropout_1 (Dropout)          (None, 2, 64)             0         
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 1, 64)             0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 64)                0         
_________________________________________________________________
dense_1 (Dense)              (None, 3)                195       
=================================================================
Total params: 13,258
Trainable params: 13,258
Non-trainable params: 0
_________________________________________________________________