CODEWITHSUNDEEP
python-3.xtensorflowkerasdeep-learning
YiPeng Li
YiPeng Li

Reputation: 1

Number of operation increases with tf.gradient

So I was trying to calculate the gradient wrt input, using a combination of Keras and tensorflow:

the code (in a loop) is like:

import keras.backend as K
loss = K.categorical_crossentropy(model's output, target)
gradient = sess.run([tf.gradients(loss, model.input, colocate_gradients_with_ops=True)],feed_dict={model.input: img}) # img is a numpy array that fits the dimension requirements

n_operations = len(tf.get_default_graph().get_operations())

I noticed that "n_operations" increases every iteration, and so as time it costs. Is that normal? Is there any way to prevent this?

Thank you!

Upvotes: 0

Views: 30

Answers (1)

Thomas Pinetz
Thomas Pinetz

Reputation: 7148

No this is not the desired behavior. Your problem is that you are defining your gradient operation again and again, while you only need to execute the operation. The tf.gradient function pushes new operations onto the graph and return a handle to those gradients. So you only have to execute them to get the desired results. With multiple runs of the function multiple operations are generated and this will eventually ruin your performance. The solution is as follows:

# outside the loop
loss = K.categorical_crossentropy(model's output, target)
gradients = tf.gradients(loss, model.input, colocate_gradients_with_ops=True)

# inside the loop
gradient_np = sess.run([gradients],feed_dict={model.input: img}) # img is a numpy array that fits the dimension requirements

Upvotes: 1

Related Questions