CODEWITHSUNDEEP
machine-learningtensorflowdeep-learning
vaj oja
vaj oja

Reputation: 1161

adding dropout in tensorflow causes incorrect result?

I use tensorflow and python to predict the stock price in a demo. But when i add dropout to the code, the generated figure doesnt seem to be correct. Please advise where is wrong.

with tf.variable_scope(scope_name):
    cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=n_inputs)
    lstm_dropout = tf.nn.rnn_cell.DropoutWrapper(cell,input_keep_prob=1.0, output_keep_prob=1.0)
    cell = tf.nn.rnn_cell.MultiRNNCell([lstm_dropout]*num_layers)
    output, state = tf.nn.rnn(cell, input, dtype=tf.float32)

enter image description here

enter image description here

Upvotes: 0

Views: 546

Answers (1)

DAlolicorn
DAlolicorn

Reputation: 106

You should only apply dropout in training, but not in inference.

You can do this by pass the dropout probability by a placeholder.

Then set the keep probability to one when inference.

As your example : 

input_keep_prob = tf.placeholder(tf.float32)
output_keep_prob = tf.placeholder(tf.float32)
with tf.variable_scope(scope_name):
    cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=n_inputs)
    lstm_dropout = tf.nn.rnn_cell.DropoutWrapper(cell,input_keep_prob=input_keep_prob, 
        output_keep_prob=output_keep_prob)
    cell = tf.nn.rnn_cell.MultiRNNCell([lstm_dropout]*num_layers)
    output, state = tf.nn.rnn(cell, input, dtype=tf.float32) 
#setup your loss and training optimizer
#y_pred = .....
#loss = .....
#train_op = .....

with tf.Session() as sess:
    sess.run(train_op, feed_dict={input_keep_prob=0.7, output_keep_prob=0.7}) #set dropout when training
    y = sess.run(y_pred, feed_dict={input_keep_prob=1.0, output_keep_prob=1.0}) #retrieve the prediction without dropout when inference

Upvotes: 1

Related Questions