CODEWITHSUNDEEP
numpymachine-learningtensorflow
ytutow
ytutow

Reputation: 385

compute Hessians w.r.t higher rank variable not work neither by tf.hessians() nor tf.gradients()

When we need to calculate double gradient or Hessian, in tensorflow, we may use tf.hessians(F(x),x), or use tf.gradient(tf.gradients(F(x),x)[0], x)[0]. However, when x is not rank one, I was told the following error when use tf.hessians().

ValueError: Cannot compute Hessian because element 0 of xs does not have rank one.. Tensor model_inputs/action:0 must have rank 1. Received rank 2, shape (?, 1)

in following code:

with tf.name_scope("1st scope"):
      self.states = tf.placeholder(tf.float32, (None, self.state_dim), name="states")
      self.action = tf.placeholder(tf.float32, (None, self.action_dim), name="action")

with tf.name_scope("2nd scope"):
  with tf.variable_scope("3rd scope"):
    self.policy_outputs = self.policy_network(self.states)
    # use tf.gradients twice
    self.actor_action_gradients = tf.gradients(self.policy_outputs, self.action)[0]
    self.actor_action_hessian = tf.gradients(self.actor_action_gradients, self.action)[0]
    # or use tf.hessians 
    self.actor_action_hessian = tf.hessian(self.policy_outputs, self.action)

When using tf.gradients(), also causes an error:

in create_variables self.actor_action_hessian = tf.gradients(self.actor_action_gradients, self.action)[0]

AttributeError: 'NoneType' object has no attribute 'dtype'

How can I fix this, does neither tf.gradients() nor tf.hessians() can be used in this case?

Upvotes: 2

Views: 326

Answers (1)

lejlot
lejlot

Reputation: 66775

The second approach is fine, error is somewhere else, namely your graph is not connected.

self.actor_action_gradients = tf.gradients(self.policy_outputs, self.action)[0]
self.actor_action_hessian = tf.gradients(self.actor_action_gradients, self.action)[0]

errror is thrown in second line because self.actor_action_gradients is None, and so you can't compute its gradient. Nothing in your code suggests that self.policy_outputs depends on self.action (and it shouldn't, since its action that depends on policy, not policy on action).

Once you fix this you will notice, that "hessian" is not really a hessian but a vector, to form a proper hessian of f wrt. x you have to iterate over all values returned by tf.gradients, and compute tf.gradients of each one independently. This is a known limitation in TF, and no simpler way is available right now.

Upvotes: 1

Related Questions