CODEWITHSUNDEEP
pythontensorflowmachine-learningneural-networklstm
kiriloff
kiriloff

Reputation: 26333

Tensorflow - building LSTM model - need for tf.keras.layers.Dense()

Python 3.7 tensorflow

I am experimenting Time series forecasting w Tensorflow

I understand the second line creates a LSTM RNN i.e. a Recurrent Neural Network of type Long Short Term Memory.

Why do we need to add a Dense(1) layer in the end? 

single_step_model = tf.keras.models.Sequential()
single_step_model.add(tf.keras.layers.LSTM(32, input_shape=x_train_single.shape[-2:]))
single_step_model.add(tf.keras.layers.Dense(1))

Tutorial for Dense() says

Dense implements the operation: output = activation(dot(input, kernel) + bias) where activation is the element-wise activation function passed as the activation argument, kernel is a weights matrix created by the layer, and bias is a bias vector created by the layer (only applicable if use_bias is True).

would you like to rephrase or elaborate on need for Dense() here ?

Upvotes: 1

Views: 2391

Answers (2)

codeblooded
codeblooded

Reputation: 350

Well in the tutorial you are following Time series forecasting, they are trying to forecast temperature (6 hrs ahead). For which they are using an LSTM followed by a Dense layer. 

single_step_model = tf.keras.models.Sequential()
single_step_model.add(tf.keras.layers.LSTM(32, input_shape=x_train_single.shape[-2:]))
single_step_model.add(tf.keras.layers.Dense(1))

Dense layer is nothing but a regular fully-connected NN layer. In this case you are bringing down the output dimensionality to 1, which should represent some proportionality (need not be linear) to the temperature you are trying to predict. There are other layers you could use as well. Check out, Keras Layers.

If you are confused about the input and output shape of LSTM, check out I/O Shape.

Upvotes: 0

josepdecid
josepdecid

Reputation: 1847

The following line

single_step_model.add(tf.keras.layers.LSTM(32, input_shape=x_train_single.shape[-2:]))

creates an LSTM layer which transforms each input step of size #features into a latent representation of size 32. You want to predict a single value so you need to convert this latent representation of size 32 into a single value. Hence, you add the following line

single_step_model.add(tf.keras.layers.Dense(1))

which adds a Dense Layer (Fully-Connected Neural Network) with one neuron in the output which, obviously, produces a single value. Look at it as a way to transform an intermediate result of higher dimensionality into the final result.

Upvotes: 2

Related Questions