How to assign a value to a TensorFlow variable?

Question

I am trying to assign a new value to a tensorflow variable in python.

import tensorflow as tf
import numpy as np

x = tf.Variable(0)
init = tf.initialize_all_variables()
sess = tf.InteractiveSession()
sess.run(init)

print(x.eval())

x.assign(1)
print(x.eval())

But the output I get is

0
0

So the value has not changed. What am I missing?

Answer 1

In TF1, the statement x.assign(1) does not actually assign the value 1 to x, but rather creates a tf.Operation that you have to explicitly run to update the variable.* A call to Operation.run() or Session.run() can be used to run the operation:

assign_op = x.assign(1)
sess.run(assign_op)  # or `assign_op.op.run()`
print(x.eval())
# ==> 1

(* In fact, it returns a tf.Tensor, corresponding to the updated value of the variable, to make it easier to chain assignments.)

However, in TF2 x.assign(1) will now assign the value eagerly:

x.assign(1)
print(x.numpy())
# ==> 1

Answer 2

I answered a similar question here. I looked in a lot of places that always created the same problem. Basically, I did not want to assign a value to the weights, but simply change the weights. The short version of the above answer is:

tf.keras.backend.set_value(tf_var, numpy_weights)

Answer 3

You can also assign a new value to a tf.Variable without adding an operation to the graph: tf.Variable.load(value, session). This function can also save you adding placeholders when assigning a value from outside the graph and it is useful in case the graph is finalized.

import tensorflow as tf
x = tf.Variable(0)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
print(sess.run(x))  # Prints 0.
x.load(1, sess)
print(sess.run(x))  # Prints 1.

Update: This is depricated in TF2 as eager execution is default and graphs are no longer exposed in the user-facing API.

Answer 4

Use Tensorflow eager execution mode which is latest.

import tensorflow as tf
tf.enable_eager_execution()
my_int_variable = tf.get_variable("my_int_variable", [1, 2, 3])
print(my_int_variable)

Answer 5

Here is the complete working example:

import numpy as np
import tensorflow as tf

w= tf.Variable(0, dtype=tf.float32) #good practice to set the type of the variable
cost = 10 + 5*w + w*w
train = tf.train.GradientDescentOptimizer(0.01).minimize(cost)

init = tf.global_variables_initializer()
session = tf.Session()
session.run(init)

print(session.run(w))

session.run(train)
print(session.run(w)) # runs one step of gradient descent

for i in range(10000):
  session.run(train)

print(session.run(w))

Note the output will be:

0.0
-0.049999997
-2.499994

This means at the very start the Variable was 0, as defined, then after just one step of gradient decent the variable was -0.049999997, and after 10.000 more steps we are reaching -2.499994 (based on our cost function).

Note: You originally used the Interactive session. Interactive session is useful when multiple different sessions needed to be run in the same script. However, I used the non interactive session for simplicity.

Answer 6

Also, it has to be noted that if you're using your_tensor.assign(), then the tf.global_variables_initializer need not be called explicitly since the assign operation does it for you in the background.

Example:

In [212]: w = tf.Variable(12)
In [213]: w_new = w.assign(34)

In [214]: with tf.Session() as sess:
     ...:     sess.run(w_new)
     ...:     print(w_new.eval())

# output
34 

However, this will not initialize all variables, but it will only initialize the variable on which assign was executed on.

Answer 7

First of all you can assign values to variables/constants just by feeding values into them the same way you do it with placeholders. So this is perfectly legal to do:

import tensorflow as tf
x = tf.Variable(0)
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print sess.run(x, feed_dict={x: 3})

Regarding your confusion with the tf.assign() operator. In TF nothing is executed before you run it inside of the session. So you always have to do something like this: op_name = tf.some_function_that_create_op(params) and then inside of the session you run sess.run(op_name). Using assign as an example you will do something like this:

import tensorflow as tf
x = tf.Variable(0)
y = tf.assign(x, 1)
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print sess.run(x)
    print sess.run(y)
    print sess.run(x)

Answer 8

There is an easier approach:

x = tf.Variable(0)
x = x + 1
print x.eval()