CODEWITHSUNDEEP
kerasnlpdeep-learninglstmrecurrent-neural-network
XIN LIU
XIN LIU

Reputation: 87

Concatenate outputs of LSTM in Keras

I intend to feed all outputs of timesteps from a LSTM to a fully-connected layer. However, the following codes fail. How can I reduce 3D output of LSTM to 2D by concatenating each output of timestep?

X = LSTM(units=128,return_sequences=True)(input_sequence)
X = Dropout(rate=0.5)(X)
X = LSTM(units=128,return_sequences=True)(X)
X = Dropout(rate=0.5)(X)
X = Concatenate()(X)
X = Dense(n_class)(X)
X = Activation('softmax')(X)

Upvotes: 0

Views: 1802

Answers (2)

ixeption
ixeption

Reputation: 2060

Additional to @todays answer: It seems like you want to use return_sequences just to concatenate it into a dense layer. If you did not already try it with return_sequeunces=False, I would recommend you to do to so. The main purpose of return_sequences is to stack LSTMS or to make seq2seq predictions. In your case it should be enough to just use the LSTM.

Upvotes: 0

today
today

Reputation: 33470

You can use the Flatten layer to flatten the 3D output of LSTM layer to a 2D shape.

As a side note, it is better to use dropout and recurrent_dropout arguments of LSTM layer instead of using Dropout layer directly with recurrent layers.

Upvotes: 1

Related Questions