CODEWITHSUNDEEP
pythontensorflow
mazecreator
mazecreator

Reputation: 555

Persistence of variables within a single session

I am working with Reinforcement Learning and would like to store my examples within the TF graph to avoid moving data between python and TF. I thought I could use a TF.Variable() as it should persist for the entire session and simply add a sample on the end when needed. Here is an example of one of the approaches:

import tensorflow as tf

initx = [[1.0, 2.0, 3.0, 4.0],
         [4.0, 5.0, 6.0, 7.0],
         [7.0, 8.0, 9.0, 8.0]]

x = tf.get_variable("x",dtype=tf.float32,validate_shape=False,initializer=tf.constant(initx))
xappend = tf.placeholder(dtype=tf.float32,shape=[None,4])
#xappend = tf.get_variable("xappend",shape=[1,4],dtype=tf.float32,validate_shape=False)

x = tf.concat(0,[x,xappend])

with tf.Session('') as session:
  session.run(tf.initialize_all_variables())    
  print session.run([x],feed_dict={xappend:[[-1.0,98.0,97.0,96.0]] })
  print session.run([x],feed_dict={xappend:[[-2.0,98.0,97.0,96.0]] })
  print session.run([x],feed_dict={xappend:[[-3.0,98.0,97.0,96.0]] })
  print session.run([x],feed_dict={xappend:[[-4.0,98.0,97.0,96.0]] })

My thought is this should concatenate the "xappend" value to the end of tensor "x" and grow as each observation is added. This is not what seems to happen. For each run, I get the initialized value plus the xappend value, just a single instance of xappend. I can't seem to get the -1,-2,-3,-4 to append to the end of the tensor within the session. Here is the output:

[array([[  1.,   2.,   3.,   4.],  
   [  4.,   5.,   6.,   7.],  
   [  7.,   8.,   9.,   8.],  
   [ -1.,  98.,  97.,  96.]], dtype=float32)]  
[array([[  1.,   2.,   3.,   4.],  
   [  4.,   5.,   6.,   7.],  
   [  7.,   8.,   9.,   8.],  
   [ -2.,  98.,  97.,  96.]], dtype=float32)]  
[array([[  1.,   2.,   3.,   4.],  
   [  4.,   5.,   6.,   7.],  
   [  7.,   8.,   9.,   8.],  
   [ -3.,  98.,  97.,  96.]], dtype=float32)]  
[array([[  1.,   2.,   3.,   4.],  
   [  4.,   5.,   6.,   7.],  
   [  7.,   8.,   9.,   8.],  
   [ -4.,  98.,  97.,  96.]], dtype=float32)]

I have tried another approach within the session and get the same unexpected results:

...
update = tf.assign(xappend, [[-1.0,98.0,97.0,96.0]], validate_shape=False)
print session.run([update])
print session.run([x])
update = tf.assign(xappend, [[-2.0,98.0,97.0,96.0]], validate_shape=False)
print session.run([update])
print session.run([x])
...

I know that when I have weights, they persist between calls within the same session. I am sure it has something to do with the size change with the tf.concat() function that doesn't allow persistence of the "x" variable in this graph.

Any ideas what could be going wrong with this approach? Is there a better approach?

Upvotes: 0

Views: 318

Answers (1)

Olivier Moindrot
Olivier Moindrot

Reputation: 28198

You were almost there. If you want to modify a Variable, you have to create an assign op. This operation will be run in sess.run() and will update the content of the Variable.

Here the trick is that when assigning the Variable, you modify its shape. You have to use tf.assign with argument validate_shape=False. This was hinted in the doc:

If you want to change the shape of a variable later you have to use an assign Op with validate_shape=False

But the documentation for tf.assign does not exist.

initx = [[1.0, 2.0, 3.0, 4.0],
         [4.0, 5.0, 6.0, 7.0],
         [7.0, 8.0, 9.0, 8.0]]

x = tf.get_variable("x",dtype=tf.float32,validate_shape=False,initializer=tf.constant(initx))
xappend = tf.placeholder(dtype=tf.float32,shape=[None,4])

new_x = tf.concat(0, [x, xappend])
append_op = tf.assign(x, new_x, validate_shape=False)

with tf.Session() as sess:
    sess.run(tf.initialize_all_variables())
    sess.run(append_op, feed_dict={xappend:[[-1.0,98.0,97.0,96.0]] })
    sess.run(append_op, feed_dict={xappend:[[-2.0,98.0,97.0,96.0]] })
    sess.run(append_op, feed_dict={xappend:[[-3.0,98.0,97.0,96.0]] })
    sess.run(append_op, feed_dict={xappend:[[-4.0,98.0,97.0,96.0]] })

    sess.run(x)  # should give you the expected result, of shape [7, 4]

However, this code will likely not be very efficient because you constantly change the shape of the Variable.

Better ways will be either to store a python list, or (better) to use a fixed size array of shape [max_length, 4] where you specify the max_lenght argument in advance, and fill the array row by row.

Upvotes: 1

Related Questions