CODEWITHSUNDEEP
tensorflow
Antonio Sesto
Antonio Sesto

Reputation: 3154

Control dependencies and order of evaluation

Please consider the following code: 

import tensorflow as tf
import numpy as np

with tf.device("gpu:0"):
    sess = tf.InteractiveSession()
    idx = tf.constant(0)
    # 10 iterations
    while_condition = lambda i: tf.less(i, tf.constant(10))        
    acc = tf.Variable(0, dtype=tf.float64)
    # the body of the while adds 1 to acc in each iteration
    def body_accumulator(i):
        mainOp = tf.assign_add(acc, 1.0)
        return tf.tuple([tf.add(i, 1)], control_inputs=[mainOp])
    whileOp = tf.while_loop(while_condition, body_accumulator, [idx])

    # My idea: return acc after evaluating whileOp, whose code modifies acc
    def f(dummy):
        with tf.control_dependencies([whileOp]):
            # with return tf.identity(acc) it works
            return acc
    def g():
        return acc

    sess.run(tf.global_variables_initializer())
    print('"g: return acc .eval()" - this is the only time where I would expect 0')
    print(g().eval())
    print('f(dummy)')
    print(f(1).eval())
    print('whileOp.eval()')
    print(whileOp.eval())
    print('acc value:')
    print(acc.eval())
    print('"g: return acc .eval()"')
    print(g().eval())

The output is:

"g: return acc .eval()" - this is the only time where I would expect 0
0.0
f(dummy)
0.0
whileOp.eval()
10
acc value:
10.0
"g: return acc .eval()"
10.0

My question is:

why does f(1).eval() return 0 even if there is a control dependency on the whileOp that modifies the returned variable acc?

After reading the documentation, I was expecting whileOp to be evaluated before returning acc. How should I write the function f(.) in order to force the evaluation of whileOp?

In f(.), if I return tf.identity(acc) instead of acc, it works as I expect.

Upvotes: 1

Views: 506

Answers (1)

Alexandre Passos
Alexandre Passos

Reputation: 5206

You're seeing problems because of the weird aliasing model for tensorflow variables. This is why things work with tf.identiy but not without.

If you enable resource variables (use tf.get_variable(..., use_resource=True) to create your variables) you will get the behavior you want.

Upvotes: 1

Related Questions