CODEWITHSUNDEEP
tensorflow
Stepan Yakovenko
Stepan Yakovenko

Reputation: 9216

Why AdamOptimizer fails to find optimal value to minimize x*x?

I am trying to minimize x*x with adagrad optimiser. I expect to get x=0 as result, but I get value x, close to initial value.

import tensorflow as tf
x=tf.Variable(-2.)
sq=x*x
o = tf.train.AdamOptimizer(1e-1).minimize(sq)
with tf.Session() as sess:
    init = tf.global_variables_initializer()
    sess.run([init])
    sess.run([o])
    r=sess.run([x])
    print("done",r)

I get -1.9 as a result, instead of expected 0.

Do I understand correctly that -2 is initial value here, or is it something else? Does AdamOptimiser perform just one step or is it possible to launch it for continious optimisation? How do I get x=0 as result?

Upvotes: 0

Views: 67

Answers (1)

Lior
Lior

Reputation: 2019

sess.run([0]) runs only a single step. To perform a full optimization, you need to run many steps, which can be done by repeating the single step in a loop.

Thus, you can replace sess.run([o]) with:

for i in range(1000):
   sess.run([o])

This yields the results 3.4735016e-23, very close to the expected 0.

In my experience, people usually run many optimization steps just as I demonstrated, with a for loop. If you are interested in implementing the loop as a TensorFlow operation, and then running this operation only once, this can be done, but it is not recommended. The reasons are: (a) I don't think you will gain any "elegance" in your code by doing this. (b) If you want to run 1000 steps, you will need to add 1000 sets of operations to your graph, and group them as one. Contrast this to needing only one set of operations.

You can see more relevant information in this question.

Upvotes: 1

Related Questions