CODEWITHSUNDEEP
machine-learningtensorflow
Anmol Jawandha
Anmol Jawandha

Reputation: 63

Using a placeholder as a tensorflow Variable (getting Error!)

I'm trying to use a placeholder in my graph as a Variable (so I can later optimise something with respect to it), but I don't know the best way to do that. I've tried this: x = tf.placeholder(tf.float32, shape = [None,1]) x_as_variable = tf.Variable(x, validate_shape = False)

but everytime I build my graph, I get an error when I try to add my loss function: train = tf.train.AdamOptimizer().minimize(MSEloss)

The error is: ValueError: as_list() is not defined on an unknown TensorShape.

Even if you're not familiar with the error entirely, I'd really appreciate it if you could guide me on how to build a replicate Variable which takes on the value of my placeholder. Thanks!

Upvotes: 1

Views: 2851

Answers (1)

mrry
mrry

Reputation: 126154

As you've noticed, TensorFlow optimizers (i.e. subclasses of tf.train.Optimizer) operate on tf.Variable objects because they need to be able to assign new values to those objects, and in TensorFlow only variables support an assign operation. If you use a tf.placeholder(), there's nothing to update, because the value of a placeholder is immutable within each step.

So how do you optimize with respect to a fed-in value? I can think of two options:

  1. Instead of feeding a tf.placeholder(), you could first assign a fed-in value to a variable and then optimize with respect to it:

    var = tf.Variable(...)
set_var_placeholder = tf.placeholder(tf.float32, ...)
set_var_op = var.assign(set_var_placeholder)
# ...
train_op = tf.train.AdamOptimizer(...).minimize(mse_loss, var_list=[var, ...])

# ...
initial_val = ...  # A NumPy array.
sess.run(set_var_op, feed_dict={set_var_placeholder: initial_val})
sess.run(train_op)
updated_val = sess.run(var)

  2. You could use the lower-level tf.gradients() function to get the gradient of the loss with respect to a placeholder in a single step. You could then use that gradient in Python:

    var = tf.placeholder(tf.float32, ...)
# ...
mse_loss = ...
var_grad, = tf.gradients(loss, [var])

var_grad_val = sess.run(var_grad, feed_dict={var: ...})

PS. The code in your question, where you define a tf.Variable(tf.placeholder(...), ...) is just defining a variable whose initial value is fed by the placeholder. This probably isn't what you want, because the training op that the optimizer creates will only use the value assigned to the variable, and ignore whatever you feed to the placeholder (after the initialization step).

Upvotes: 4

Related Questions