CODEWITHSUNDEEP
pythontensorflowgraphautograd
Broono
Broono

Reputation: 111

tf.gradients() won't work with 'tf.assign()', but works with '='

In a simple code below, gradient gets computed correctly.

import tensorflow as tf

x = tf.constant([1, 2, 3, 4], dtype=tf.float32)
y = tf.Variable(tf.ones_like(x), dtype=tf.float32)

y = 2*x
grad = tf.gradients(y, x)

ini = tf.global_variables_initializer()


with tf.Session() as ses:
    ses.run(ini)
    print(ses.run(grad))

The result, as expected, is [array([ 2., 2., 2., 2.], dtype=float32)]. I get into problem when trying to use tf.assign for function computation. The below code:

import tensorflow as tf

x = tf.constant([1, 2, 3, 4], dtype=tf.float32)
y = tf.Variable(tf.ones_like(x), dtype=tf.float32)

func = tf.assign(y, 2*x)
grad = tf.gradients(y, x)

ini = tf.global_variables_initializer()

with tf.Session() as ses:
    ses.run(ini)
    ses.run(func)
    print(ses.run(grad))

... yields an error:

TypeError: Fetch argument None has invalid type <class 'NoneType'>.

Why is that so? Is the connection between x and y node somehow "lost" via the tf.assign operation?

Upvotes: 1

Views: 558

Answers (1)

Maxim
Maxim

Reputation: 53758

In the second example, there's no dependency between x and y. func is an op that depends on both and happens to modify y. If you inspect the corresponding tf.assign op, you'll see:

op: "Assign"
input: "Variable"   # this is y
input: "mul"        # this is 2*x

But x and y are independent, that's why the engine fails to take the gradient.

Upvotes: 1

Related Questions