Understanding connecting a Dense layer to LSTM

Question

I am unable to correctly connect my dense layer to my LSTM layers. My Y values range from 0-1 so sigmoid seems logical to me.

I get the error:

Error when checking target: expected dense_4 to have 2 dimensions, but got array with shape (993, 300, 1)

To me it seems i have the input shape correct total DF is (350700 , 2413) which i reshape to a ( 1169 , 300 , 2413 ) // not including Y value. I just can't seem to figure out how to get the dense layer working and apply the sigmoid to my Y.

With the train test split i have a y_train of (993, 300, 1) which is the main issue of my error but i can't seem to understand what i have done wrong. x_train is ( 933, 300, 2413) x_test = ( 176, 300 , 2413) y_test= (176, 300, 1)

Here is the network i have set up. backend tensorflow (also used theano same issue)

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
lstm_21 (LSTM)               (None, 300, 1000)         13656000  
_________________________________________________________________
lstm_22 (LSTM)               (None, 300, 500)          3002000   
_________________________________________________________________
lstm_23 (LSTM)               (None, 300, 250)          751000    
_________________________________________________________________
lstm_24 (LSTM)               (None, 300, 100)          140400    
_________________________________________________________________
lstm_25 (LSTM)               (None, 50)                30200     
_________________________________________________________________
dense_4 (Dense)              (None, 1)                 51        
=================================================================
Total params: 17,579,651
Trainable params: 17,579,651
Non-trainable params: 0
_________________________________________________________________

here is a my code.

import numpy as np
import pandas as pd
from keras.models import Sequential
from keras.layers.advanced_activations import LeakyReLU
from keras.layers import Dense, Activation, LSTM, Flatten
from keras import backend as K
from sklearn.model_selection import train_test_split

aa = aa[np.isfinite(aa['Y1'])]
aa=aa[-350700:]

Y=aa['Y1'].values.reshape(1169,300,1) #break into 1169 samples @ 300 timestamps
aa.drop(drop1, axis=1, inplace=True) #drop the Y1 feature and others not needed.


features=aa.shape[1]

X=aa.values.reshape(1169,300,features)

seed = 7

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.15, random_state=seed)

model = Sequential()
model.add(LSTM(1000, input_shape=(300,features),activation='relu',return_sequences=True))
model.add(LSTM(500,activation='relu',return_sequences=True))
model.add(LSTM(250,activation='relu',return_sequences=True))
model.add(LSTM(100, activation='relu',return_sequences=True))
model.add(LSTM(50,activation='relu',return_sequences=False))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss='mae',
              optimizer='adam',
              metrics=['mse', 'mae', 'mape'])

print(model.summary())


# evaluate model with standardized dataset
model.fit(X_train, y_train, validation_data=(X_test,y_test), epochs=15000)

Answer 1

Your "data" is not compatible with your "last layer shape".

• Either you need Y_train with shape (993,1) - Classifying the entire sequence
• Or you need to keep return_sequences=True in "all" LSTM layers - Classifying each time step

What is correct depends you what you're trying to do.