CODEWITHSUNDEEP
pythontensorflowstatelstm
Chris
Chris

Reputation: 61

When the LSTM state is reset

I'm trying to implement a LSTM neural network using tensorflow to do keyword detection. I fed the neural network with sequences of 400ms. However, during training, I don't want the LSTM to remember the sequence 1 while trying to learn the sequence 6 for instance. So how can I reset the state of the LSTM during the training. Does the initial_state in argument of outputs, state = rnn.rnn(cell, inputs, initial_state=self._initial_state) allows to reset the memory of the LSTM once the entire batch is fed?

I tried to understand the implementation with this link:

https://github.com/tensorflow/models/blob/master/tutorials/rnn/ptb/ptb_word_lm.py

Upvotes: 1

Views: 3312

Answers (1)

Kilian Obermeier
Kilian Obermeier

Reputation: 7118

In the ptb_word_lm.py self._initial_state is set only once in the whole program:

self._initial_state = cell.zero_state(batch_size, data_type())

This means, it remains a constant zero vector. So the initial state for the unrolling of the LSTM is always zero. You don't need to explicitly reset the memory after a batch is fed.

If you would like to manually update the LSTM's state / self._initial_state you need to define it as a Variable instead of a Tensor. See the answers here for more info.

Upvotes: 2

Related Questions