CODEWITHSUNDEEP
pythonvariablestensorflowassigncontrol-flow
Simon.He
Simon.He

Reputation: 43

How to use the old value and the new value of a Variable in Tensorflow?

I want to use the old value and the new value of a Variable. But I am confused about when the Assign Op is applied.

Here is a simple example. The outputs of output and output2 are different.

v = tf.Variable(0)
with tf.Session() as sess:
    sess.run(v.initializer)

    new_v = v.assign(v + 10)
    output = v + 0 # `v` is evaluated before the assignment ?
    output2 = v  # `v` is evaluated after the assignment ?

    print(sess.run([ output, output2, new_v])) 
    print(sess.run(output))

The result is

[0, 10, 10]

10

Please advise me what is the proper way to use the old value and the new value of a Variable. Thanks.

According to chrisz's answer, I tried tf.control_dependencies to get the old value of v. But the result is not what I expected. I still need to add 0 to v to get the old value.

Here is the test code. And I add a 0 to get the same result as above. Otherwise, the result of output_old will be 10

v = tf.Variable(0)
with tf.Session() as sess:
    sess.run(v.initializer)
    output_old = v + 0         # If I want the old value, this extra add is needed
    with tf.control_dependencies([output_old]):
        new_v = v.assign(v + 10)
        output_new = new_v

    print(sess.run([output_old, output_new, new_v])) 
    print(sess.run(output_old))

Upvotes: 4

Views: 373

Answers (1)

Maxim
Maxim

Reputation: 53768

First up, when the fetches are independent, there's in general no guarantee in which order they are calculated (a question with a similar case).

E.g., if you run your script in a loop, you may occasionally get [10, 10, 10], which would mean that new_v is evaluated first. On my machine, I couldn't make output2 evaluate to 0, so it's most probably implementation feature, but I won't be surprised if it changes from platform to platform or from version to version. The only guaranteed value is new_v which will always be 10; both output and output2 can be either 0 or 10.

To answer your question, the best way to get both values simultaneously is to add another variable and use tf.control_dependencies context:

v = tf.Variable(0)
tmp = tf.Variable(0)
with tf.Session() as sess:
    sess.run([v.initializer, tmp.initializer])
    saved_v = tmp.assign(v)
    with tf.control_dependencies([saved_v]):
      new_v = v.assign(v + 10)
      old_v = tf.identity(saved_v)
      # At this point, `new_v` is guaranteed to be 10, 
      # `old_v` and `saved_v` are guaranteed to be 0

Note that it won't in general work without tf.control_dependencies, as old_v and saved_v can be evaluated after new_v. Your trick with a dependency on output_old = v + 0 also works, but it looks more like a hack to me. In any case, you can't get avoid tf.control_dependencies.

Upvotes: 2

Related Questions