CODEWITHSUNDEEP
machine-learningneural-networkkeraslstmrecurrent-neural-network
Mahdi K
Mahdi K

Reputation: 97

Dimension Error while using LSTM after Convolutional neural network in Keras

In keras 1.2.2, I have made a dataset which has these dimensions:

X_train: (2000, 100, 32, 32, 3) 
y_train: (2000,1)

Here, 2000 is the number of instances (batches of data), 100 is the number of samples in each batch, 32 is the image rows and cols, and 3 is the number of channels (RGB).

I have written this code which applies an LSTM after a CNN, however, I get this error:

ValueError: Input 0 is incompatible with layer lstm_layer: expected ndim=3, found ndim=2

This is my code:

import keras
from keras.layers import Input ,Dense, Dropout, Activation, LSTM
from keras.layers import Convolution2D, MaxPooling2D, Flatten, Reshape
from keras.models import Sequential
from keras.layers.wrappers import TimeDistributed
from keras.layers.pooling import GlobalAveragePooling1D
from keras.optimizers import SGD
from keras.utils import np_utils
from keras.models import Model


import numpy as np

timesteps=100;
number_of_samples=2500;
nb_samples=number_of_samples;
frame_row=32;
frame_col=32;
channels=3;

nb_epoch=1;
batch_size=timesteps;

data= np.random.random((2500,timesteps,frame_row,frame_col,channels))
label=np.random.random((2500,timesteps,1))

X_train=data[0:2000,:]
y_train=label[0:2000]

X_test=data[2000:,:]
y_test=label[2000:,:]

#%%

model=Sequential();                          

model.add(Convolution2D(32, 3, 3, border_mode='same',
                        input_shape=X_train.shape[2:]))
model.add(Activation('relu'))
model.add(Convolution2D(32, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Convolution2D(64, 3, 3, border_mode='same'))
model.add(Activation('relu'))
model.add(Convolution2D(64, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))


model.add(Dense(35, input_shape=(timesteps,512), name="first_dense" ));
#model.add(Dense(1, name="test_dense"));         

model.add(LSTM(20, return_sequences=True, name="lstm_layer"));

#%%
model.add(TimeDistributed(Dense(1), name="time_distr_dense_one"))
model.add(GlobalAveragePooling1D(name="global_avg"))

#%%

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])
#%%
model.fit(X_train, y_train,
          batch_size=batch_size,
          nb_epoch=nb_epoch,
          validation_data=(X_test, y_test))

Upvotes: 5

Views: 783

Answers (3)

Emre Tatbak
Emre Tatbak

Reputation: 113

Output dimension of LSTM should be the same with your timesteps. Try this:

model.add(LSTM(output_dim=timesteps, return_sequences=True))

It works for me.

Upvotes: 0

Marcin Możejko
Marcin Możejko

Reputation: 40506

Try exchanging each Convolution2D(...) to:

TimeDistributed(Convolution2D(...))

You need to let your model know that your data is sequential and you want to apply some layer to each element in your sequence. This is what TimeDistributed wrapper is for.

Upvotes: 4

Nassim Ben
Nassim Ben

Reputation: 11553

Are you sure that fit() is the right method to use? I would use train_on_batch since your training data is already split in batches.

To better control your input_shape, I would advise you to explicitely define it in your first layer. Instead of X_train.shape[1:], use (32,32,3) to avoid any surprises.

Does this help you?

Upvotes: 0

Related Questions