Does TensorFlow recompute nodes even if there are exactly the same computations already defined elsewhere in the graph?

Question

Consider for example this example:

train_op = opt.minimize(loss)
gradients = tf.gradients(loss, tf.trainable_variables())

Are the gradients computed twice or just once?

Or this example:

a = y + z
b = y + z

Is the addition y + z computed twice or just once?

Answer 1

It is computed only once. See this post for more info about this and other optimizations tensorflow does at runtime.