CODEWITHSUNDEEP
pythontensorflowkerasdeep-learningloss-function
shiva
shiva

Reputation: 1189

Keras version of the combined cross-entropy and calibration loss

I recently read a paper entitled "Improved Trainable Calibration Method for Neural Networks on Medical Imaging Classification". The study incorporates calibration into the deep learning model training process by measuring the difference between predicted confidence and accuracy (DCA) and adding it as an auxiliary term to the cross entropy loss. The GitHub code is available at https://github.com/GB-TonyLiang/DCA. The DCA term is said to apply to apply a penalty when the cross-entropy loss reduces but the accuracy is plateaued. The code in Pytorch is given below:

import torch
from torch.nn import functional as F

def cross_entropy_with_dca_loss(logits, labels, weights=None, alpha=1., beta=10.):        
    ce = F.cross_entropy(logits, labels, weight=weights)

    softmaxes = F.softmax(logits, dim=1)
    confidences, predictions = torch.max(softmaxes, 1)
    accuracies = predictions.eq(labels)
    mean_conf = confidences.float().mean()
    acc = accuracies.float().sum()/len(accuracies)
    dca = torch.abs(mean_conf-acc)
    loss = alpha*ce+beta*dca
    
    return loss

I need assistance in converting this as a custom function in Keras and use it in place for categorical cross-entropy loss for multi-class classification that uses the true labels (y_true) and predicted probabilities (y_pred) and not the logits.

Upvotes: 0

Views: 498

Answers (2)

Sdhir
Sdhir

Reputation: 151

This code snippet can take true labels and predicted probabilities. y_pred is prob tensor. No need to use softmax function.

import tensorflow as tf
from keras.metrics import CategoricalAccuracy
from keras.losses import CategoricalCrossentropy

# Assuming y_pred is prob tensor, y_true is one-hot encoded
def cross_entropy_with_dca_loss(y_true, y_pred, alpha=1., beta=10.):        
    ce = CategoricalCrossentropy(from_logits=False)(y_true,y_pred)
    predictions = tf.math.argmax(y_pred, axis=1)
    confidences = tf.reduce_max(y_pred, axis=1)
    mean_conf = tf.reduce_mean(confidences)
    acc_m = CategoricalAccuracy()
    acc_m.update_state(y_true, y_pred)
    acc = acc_m.result().numpy()
    dca = tf.abs(mean_conf-acc)
    loss = alpha*ce+beta*dca
    return loss

# test on a sample data
y_true = tf.constant([[0, 1, 0], [0, 0, 1]])
y_pred = tf.constant([[0.05, 0.95, 0], [0.1, 0.8, 0.1]])
L = cross_entropy_with_dca_loss(y_true, y_pred)
print("loss", L.numpy())

Upvotes: 1

anil kumar
anil kumar

Reputation: 860

Below code is a probable equivalent of the above PyTorch code in Keras.

Except for the weights parameter. The below snippet could be helpful to you.

Kindly check the outputs. If something is wrong. Share your comments if any.

import tensorflow as tf
from keras.losses import CategoricalCrossentropy
from keras.activations import softmax

def cross_entropy_with_dca_loss(logits, labels, weights=None, alpha=1., beta=10.):
    cce = CategoricalCrossentropy() 
    ce = cce(logits, labels) # not sure about weights parameter.
    softmaxes = softmax(logits, axis=1)
    confidences = tf.reduce_max(softmaxes, axis=1)
    mean_conf = tf.reduce_mean(confidences)
    acc = tf.reduce_mean(tf.cast(tf.equal(logits, labels), dtype=tf.float32))
    dca = tf.abs(mean_conf - acc)
    loss = alpha * ce + beta * dca
    return loss

Upvotes: 1

Related Questions