CODEWITHSUNDEEP
pythonneural-networktensorflowrecurrent-neural-networklstm
agupta231
agupta231

Reputation: 1171

Tensorflow LSTM network outputting the same values for whole batch

I'm running a RNN on tensorflow, in which my output is a very simple 2 vector array. All the RNN has to do is to choose one value to make 1 and one to make 0. However, when I run my batch through the RNN, I am appearing to get the same output for every input of the batch. For example:

If my actual output labels are:

[[ 1.  0.]
 [ 1.  0.]
 [ 0.  1.]
 [ 1.  0.]
 [ 1.  0.]
 [ 1.  0.]
 [ 1.  0.]
 [ 1.  0.]
 [ 1.  0.]
 [ 1.  0.]
 [ 1.  0.]
 [ 1.  0.]
 [ 1.  0.]
 [ 0.  1.]
 [ 0.  1.]
 [ 1.  0.]
 [ 1.  0.]
 [ 0.  1.]
 [ 1.  0.]
 [ 1.  0.]
 [ 1.  0.]
 [ 1.  0.]]

My RNN outputs:

[[  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]
 [  1.00000000e+00   7.93261263e-14]]

Which is clearly wrong. I'm feeding my RNN with a tensor of shape [batch_size, time_steps_features] and I'm using tf.nn.dynamic_rnn() to run the RNN. Nothing I have tried seems to work.

I have been ripping my hair out on this, and any help would be greatly appreciated.

Thanks!

Upvotes: 1

Views: 1179

Answers (1)

Philiphorst
Philiphorst

Reputation: 45

It looks like you might not re-shuffle the samples in your batches between epochs. This can lead the minimization (of the error) to a local minimum which you won't get out of.

Try not to provide the input in the same order in every iteration and you should converge to something more sensible.

Upvotes: 4

Related Questions