CODEWITHSUNDEEP
pythonmachine-learningkerastensorflow2.0tf.keras
machinery
machinery

Reputation: 6290

No gradients provided for any variable for custom loss function

I have created a custom loss function in Keras as follows:

import tensorflow as tf
import numpy as np
def custom_loss(y_true, y_pred):
   cce = tf.keras.losses.CategoricalCrossentropy()
   loss = cce(y_true, y_pred).numpy()

   epsilon = np.finfo(np.float32).eps
   confidence = np.clip(y_true.numpy(), epsilon, 1.-epsilon)
   sample_entropy = -1. * np.sum(np.multiply(confidence, np.log(confidence) / np.log(np.e)), axis=-1)
   entropy = np.mean(sample_entropy)
   penalty = 0.1 * -entropy
   
   return loss + penalty

When I use this custom loss function I'm getting the error message

ValueError: No gradients provided for any variable

Somehow now gradients can be calculated. How needs the loss function be changed so that gradients can be calculated?

Upvotes: 1

Views: 968

Answers (1)

Mr. For Example
Mr. For Example

Reputation: 4313

Tensorflow need tensor to store the dependency info to let gradients flow backwards, if you convert tensor to numpy array in loss function then you break this dependency thus no gradients provided for any variable, so you need change every np operation in loss function to corresponding tf or backend operation, e.g:

import tensorflow as tf
import numpy as np

cce = tf.keras.losses.CategoricalCrossentropy()
epsilon = np.finfo(np.float32).eps

def custom_loss(y_true, y_pred):
    loss = cce(y_true, y_pred)
    confidence = tf.clip_by_value(y_true, epsilon, 1.-epsilon)
    sample_entropy = -1. * tf.reduce_sum(tf.math.multiply(confidence, tf.math.log(confidence) / tf.math.log(np.e)), axis=-1)
    entropy = tf.reduce_mean(sample_entropy)
    penalty = 0.1 * -entropy
    return loss + penalty

Upvotes: 1

Related Questions