CODEWITHSUNDEEP
tensorflow
randomizer
randomizer

Reputation: 845

How can I copy a variable in tensorflow

In numpy I can create a copy of the variable with numpy.copy. Is there a similar method, that I can use to create a copy of a Tensor in TensorFlow?

Upvotes: 27

Views: 61307

Answers (4)

Himanshu Tanwar
Himanshu Tanwar

Reputation: 336

In TF2 : tf.identity() will do the good deed for you. Recently I encountered some problems using the function in google colab. In case that's why you're here, this will be helping you.

Error : Failed copying input tensor from /job:localhost/replica:0/task:0/device:CPU:0 to /job:localhost/replica:0/task:0/device:GPU:0 in order to run Identity: No unary variant device copy function found for direction: 1 and Variant type_index: tensorflow::data::(anonymous namespace)::DatasetVariantWrapper [Op:Identity]

#Erroneous code
tensor1 = tf.data.Dataset.from_tensor_slices([[[1], [2]], [[3], [4]]])
tensor2 = tf.identity(tensor1)

#Correction
tensor1 = tf.data.Dataset.from_tensor_slices([[[1], [2]], [[3], [4]]])
with tf.device('CPU'): tensor2 = tf.identity(tensor1)

Upvotes: 0

dga
dga

Reputation: 21917

You asked how to copy a variable in the title, but how to copy a tensor in the question. Let's look at the different possible answers.

(1) You want to create a tensor that has the same value that is currently stored in a variable that we'll call var.

tensor = tf.identity(var)

But remember, 'tensor' is a graph node that will have that value when evaluated, and any time you evaluate it, it will grab the current value of var. You can play around with control flow ops such as with_dependencies() to see the ordering of updates to the variable and the timing of the identity.

(2) You want to create another variable and set its value to the value currently stored in a variable:

import tensorflow as tf
var = tf.Variable(0.9)
var2 = tf.Variable(0.0)
copy_first_variable = var2.assign(var)
init = tf.initialize_all_variables()
sess = tf.Session()

sess.run(init)

print sess.run(var2)
sess.run(copy_first_variable)
print sess.run(var2)

(3) You want to define a variable and set its starting value to the same thing you already initialized a variable to (this is what nivwu.. above answered):

var2 = tf.Variable(var.initialized_value())

var2 will get initialized when you call tf.initialize_all_variables. You can't use this to copy var after you've already initialized the graph and started running things.

Upvotes: 41

Salvador Dali
Salvador Dali

Reputation: 222531

You can do this in a couple of ways.

  • this will create you a copy: v2 = tf.Variable(v1)
  • you can also use identity op: v2 = tf.identity(v1) (which I think is a proper way of doing it.

Here is a code example:

import tensorflow as tf

v1 = tf.Variable([[1, 2], [3, 4]])
v_copy1 = tf.Variable(v1)
v_copy2 = tf.identity(v1)

init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)
a, b = sess.run([v_copy1, v_copy2])
sess.close()

print a
print b

Both of them would print the same tensors.

Upvotes: 10

siemanko
siemanko

Reputation: 1409

This performs a deep copy

copied_variable = tf.Variable(source_variable.initialized_value())

It also handles intialization properly, i.e. 

tf.intialize_all_variables()

will properly initialize source_variable first and then copy that value to copied_variable

Upvotes: 6

Related Questions