CODEWITHSUNDEEP
tensorflowdeep-learningconv-neural-networkgradient-descent
amankedia
amankedia

Reputation: 377

How to compute grads_and_vars with GradientDescentOptimizer

I have the following CNN code implementation.

optimizer = tf.train.GradientDescentOptimizer(cnn.learning_rate).minimize(loss = cnn.loss, global_step=global_step)
grads_and_vars = optimizer.compute_gradients(cnn.loss)
train_op = optimizer.apply_gradients(grads_and_vars, global_step=global_step)

Using it, I get the following error:

grads_and_vars = optimizer.compute_gradients(cnn.loss)
AttributeError: 'Operation' object has no attribute 'compute_gradients'

I noticed that using the above GradientDescentOptimizer implementation with tf.train.exponential_decay does not provide the attribute 'compute_gradients' for the optimizer.

Can someone help me with this?

Upvotes: 3

Views: 1465

Answers (2)

haoduohou
haoduohou

Reputation: 1

Feel free to look at the source code of optimizer.py in tensorflow (path: /tensorflow/tensorflow/python/training/optimizer.py) and you will get the answer.

The code snippet, i.e. from line 365 to line 423 is the implementation of .minimize() function. Don't worry, most of these are notes. In actually, it performs .compute_gradients() and .apply_gradients() together. So if you want to call .compute_gradients() alone, you have to ensure the variable is an instance of tf.train.GradientDescentOptimizer().

Upvotes: 0

grovina
grovina

Reputation: 3077

Both compute_gradients and apply_gradients are methods of the tf.train.GradientDescentOptimizer class.

The problem is that you are defining your optimizer as the step. You should remove the minimize call, like this:

optimizer = tf.train.GradientDescentOptimizer(cnn.learning_rate)
grads_and_vars = optimizer.compute_gradients(cnn.loss)
train_op = optimizer.apply_gradients(grads_and_vars, global_step=global_step)

and it should work.

However, if you are not doing any specific use of the gradients, you can indeed use .minimize(...) to directly define a training step, similarly to your train_op:

optimizer = tf.train.GradientDescentOptimizer(cnn.learning_rate)
train_op = optimizer.minimize(loss=cnn.loss, global_step=global_step)

Further explanation on how apply_gradients works here.

Upvotes: 1

Related Questions