CODEWITHSUNDEEP
pythontensorflowgradient-descentgradienttape
Abhimanyu Pallavi Sudhir
Abhimanyu Pallavi Sudhir

Reputation: 202

Tensorflow: correct way to initialize concatenated tensors?

Here's a MWE for the problem I'm facing:

import tensorflow as tf

with tf.GradientTape() as tape:
  x = tf.Variable(0.0)
  y = tf.Variable(x)
  z = x

print(tape.gradient(y, x))
# None

print(tape.gradient(z, x))
# 1.0

Well, obviously this is is easy to fix in this particular situation, but in the actual use case I'm facing, having to do with Recurrent Neural Networks, I need to use tf.Variable to form tensors from concatenating other tensors, like so:

Dout = tf.Variable([seed]) # initialize 
for i in range(n):
  Dout = tf.concat([Dout, 
                    G.forward_step(Dout[-1])], 
                   axis = 0)

Well, I'm fairly new to actually manipulating tensors in TF, and perhaps there's a correct way to create tensors from concatenation.

Help?

Upvotes: 1

Views: 202

Answers (1)

Abhimanyu Pallavi Sudhir
Abhimanyu Pallavi Sudhir

Reputation: 202

OK, got it -- you should be initializing as a list (not a tensor at all), then using tf.stack to convert it into a tensor.

In any case, tf.Variable is the wrong thing to use here -- we don't want a trainable variable, we want tf.constant.

Upvotes: 1

Related Questions