CODEWITHSUNDEEP
pythontensorflow
AutumnCape
AutumnCape

Reputation: 3

Tensorflow gradienttape calculating gradients by giving values 1 by 1 rather than as a batch

When I calculate the gradient of x^2 in the following way. 

x = tf.convert_to_tensor(np.linspace(-5,5,11))
y=np.zeros(10)
with tf.GradientTape() as g:
   g.watch(x)
   y=x*x
y = tf.convert_to_tensor(y)
x=tf.convert_to_tensor(x)
dy_dx = g.gradient(y, x)
print(dy_dx)

I get the values of the gradient as expected.

tf.Tensor([-10.  -8.  -6.  -4.  -2.   0.   2.   4.   6.   8.  10.], shape=(11,), dtype=float64)

I would instead like to break up the calculation and not use tensor multiplication. As I may not necessarily know the value of x before the for loop.

Naively I tried

x=tf.Variable(-5.0)
y=tf.Variable(0.0)
with tf.GradientTape(persistent=True) as g:
  g.watch(x)
  for i in range(10):
      y.assign(x*x)
      x.assign_add(1.0)

dy_dx = g.gradient(y, x)
print(dy_dx)

But this just returns [NONE].

Is there a way to separate the calculation and use a for loop instead of doing all the calculations in 1 go like the first example?

Upvotes: 0

Views: 124

Answers (1)

Nicolas Gervais
Nicolas Gervais

Reputation: 36684

You could do the loop outside of the with statement, zipping the inputs and labels:

x = tf.convert_to_tensor(np.linspace(-5,5,10))
y=np.zeros(10)

for inp, out in zip(x, y):
    with tf.GradientTape() as g:
       g.watch(inp)
       out=inp*inp

    out = tf.convert_to_tensor(out)
    inp=tf.convert_to_tensor(inp)
    dy_dx = g.gradient(out, inp)
    print(dy_dx.numpy())

-10.0
-7.777777777777778
-5.555555555555555
-3.333333333333333
-1.1111111111111107
1.1111111111111107
3.333333333333334
5.555555555555557
7.777777777777779
10.0

Upvotes: 0

Related Questions