CODEWITHSUNDEEP
pythontensorflowdeep-learning
figs_and_nuts
figs_and_nuts

Reputation: 5743

Understanding ResourceVariables in tensorflow

From here

Unlike tf.Variable, a tf.ResourceVariable has well-defined semantics. Each usage of a ResourceVariable in a TensorFlow graph adds a read_value operation to the graph. The Tensors returned by a read_value operation are guaranteed to see all modifications to the value of the variable which happen in any operation on which the read_value depends on (either directly, indirectly, or via a control dependency) and guaranteed to not see any modification to the value of the variable on which the read_value operation does not depend on. For example, if there is more than one assignment to a ResourceVariable in a single session.run call there is a well-defined value for each operation which uses the variable's value if the assignments and the read are connected by edges in the graph.

So i tried to test the behavior. My code:

tf.reset_default_graph()
a = tf.placeholder(dtype=tf.float32,shape=(), name='a')
d = tf.placeholder(dtype=tf.float32,shape=(), name='d')
b = tf.get_variable(name='b', initializer=tf.zeros_like(d), use_resource=True)
c=a+b
b_init = tf.assign(b, d)
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())   
    print(sess.run([c,b_init,b], feed_dict={a:5.,d:10.}))

This prints [15.,10.,10.]. As per my understanding of resource variables in tensorflow variable c should not have access to the value of b that was assigned to it in b_init which, would mean the output instead should be [5.,10.,0.]. Please, help me understand where i am going wrong

Upvotes: 1

Views: 1838

Answers (1)

MPękalski
MPękalski

Reputation: 7103

Two remarks:

  1. The order in which you write variables/ops in the first argument of sess.run does not mean that is the order of execution.

  2. If something worked in one step it does not mean it will work if you add loads of parallelism.

The answer to the question:

The key in the definition is depends on : a read_value operation are guaranteed to see all modifications on which the read_value depends on. If you look at the graph below, the add operation actually contains a ReadVariableOp operation for b, and then ReadVariableOp also depends on AssignVariableOp. Hence, c should take into account all modifications to b.

Unless I am mixing something, but I sound convincing to myself. :) enter image description here

If you want to see [10.0, 5.0, 0.0] you have to add tf.control_dependency like below

tf.reset_default_graph()
a = tf.placeholder(dtype=tf.float32,shape=(), name='a')
d = tf.placeholder(dtype=tf.float32,shape=(), name='d')
b = tf.get_variable(name='b', initializer=tf.zeros_like(d), use_resource=True)
c=a+b
with tf.control_dependencies([c]):
  b_init = tf.assign(b, d)


with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())   
    print(sess.run([b_init,c,b], feed_dict={a:5.,d:10.}))

Then the graph will change a bit enter image description here

Upvotes: 2

Related Questions