CODEWITHSUNDEEP
neural-networkdeep-learninglstmrecurrent-neural-network
Mohammad Diab
Mohammad Diab

Reputation: 109

How many are LSTM Gates?

It is very confusing while googling for LSTM gates , I find that some references and articles say that they are three gates: input , forget and output gates , While others say that they are four gates : learn , forget , remember, and use gate ?

So, what is the right ?

Upvotes: 2

Views: 2242

Answers (2)

Lerner Zhang
Lerner Zhang

Reputation: 7130

There are four gates: input modulation gate, input gate, forget gate and output gate, representing four sets of parameters.

We can see that there are four sets of parameters(8 matrices) colored in blue in the below graph of LSTM where f stands for the forget gate, g and i the add gate, o the output gate. Since the add gate needs two sets of parameters we can combine them as just one gate.

enter image description here

Reference:
Speech and Language Processing

Upvotes: 1

Bill Chen
Bill Chen

Reputation: 1749

Long Short Term Memory networks – usually just called “LSTMs” – are a special kind of RNN, capable of learning long-term dependencies. So there are different types of LSTMs, LSTMs have chain-like structure like RNN, but the repeating module has a different structure. Instead of having a single neural network layer, there are four, interacting in a very special way. However, there are many variants of LSTMS.

Ref: http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Upvotes: 0

Related Questions