CODEWITHSUNDEEP
tensorflowmathematical-optimizationderivative
Guillaume Chevalier
Guillaume Chevalier

Reputation: 10888

tf.hessians() - ValueError: None values not supported

Here is the traceback:

Traceback (most recent call last):
  File "test.py", line 39, in <module>
    hess = tf.hessians(loss, wrt_variables)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/ops/gradients_impl.py", line 970, in hessians
    _gradients = array_ops.unstack(_gradients)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/ops/array_ops.py", line 952, in unstack
    value = ops.convert_to_tensor(value)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 639, in convert_to_tensor
    as_ref=False)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 704, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/constant_op.py", line 113, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/constant_op.py", line 102, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/tensor_util.py", line 360, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

Variables: 

import tensorflow as tf

data_x = [0., 1., 2.]
data_y = [-1., 1., 3.]
batch_size = len(data_x)

x = tf.placeholder(shape=[batch_size], dtype=tf.float32, name="x")
y = tf.placeholder(shape=[batch_size], dtype=tf.float32, name="y")

W = tf.Variable(tf.ones(shape=[1]), dtype=tf.float32, name="W")
b = tf.Variable(tf.zeros(shape=[1]), dtype=tf.float32, name="b")

pred = x * W + b
loss = tf.reduce_mean(0.5 * (y - pred)**2)

Then, following up with this code would work:

wrt_variables = [W, b]
hess = tf.hessians(loss, wrt_variables)

But this fails:

wrt_variables = tf.concat([W, b], axis=0)
hess = tf.hessians(loss, wrt_variables)

And this would also fail:

wrt_variables = [tf.concat([W, b], axis=0)]
hess = tf.hessians(loss, wrt_variables)

It also fails for reshape operations.

The complete version of this code and with comments can be seen here: https://gist.github.com/guillaume-chevalier/6b01c4e43a123abf8db69fa97532993f

Thanks!

Upvotes: 1

Views: 1081

Answers (1)

P-Gn
P-Gn

Reputation: 24581

This is because in your graph, the node loss does not depend on the node tf.concat([W,b], axis=0). There is no backpropagation of one onto the other, and therefore no derivative.

Tensorflow is not a formal calculus engine, it can only estimate derivatives of a node by another node if the former is downstream of the later. So for example even

tf.hessian(loss, 2*W)

will fail for the same reasons (2*W is a new node and loss does not depend on it) even though the relationship to tf.hessian(loss, W) is straightfoward.

Note that the sitatuation is the same with tf.gradients, even though it fails differently: it returns Nones rather than throwing an exception.

Upvotes: 1

Related Questions