CODEWITHSUNDEEP
tensorflowkerasmetrics
Aghie
Aghie

Reputation: 81

How to define a custom accuracy in Keras to ignore samples with a particular gold label?

I want to write in Keras a custom metric (I am using the tensorflow backend) equivalent to categorical_accuracy, but where the output for samples with a particular gold label (in my case 0, from y_true) have to be ignored. For example, if my outputs were:

Pred 1 - Gold 0

Pred 1 - Gold 1

The accuracy would be 1, since samples with the gold label 0 have to be ignored. That said, the function I wrote (and that is not giving the expected results) is:

def my_accuracy(y_true, y_pred):

    mask = K.any(K.not_equal(K.argmax(y_true, axis=-1), 0), axis=-1, keepdims=True)

    masked_y_true = y_true*K.cast(mask, K.dtype(y_true))
    masked_y_pred = y_pred*K.cast(mask, K.dtype(y_pred))

    return keras.metrics.categorical_accuracy(masked_y_true, masked_y_pred)`

Any help is appreciated, thanks!

Upvotes: 2

Views: 1738

Answers (2)

Nikhil Gupta
Nikhil Gupta

Reputation: 1486

Based on @RKO's response, here is the TensorFlow 2.0 version

import tensorflow.keras as K
def ignore_acc(y_true_class, y_pred_class, class_to_ignore=0):
    ignore_mask = K.backend.cast(K.backend.not_equal(y_pred_class, class_to_ignore), 'int32')
    matches = K.backend.cast(K.backend.equal(y_true_class, y_pred_class), 'int32') * ignore_mask
    accuracy = K.backend.sum(matches) / K.backend.maximum(K.backend.sum(ignore_mask), 1)
    return accuracy.numpy()

Upvotes: 0

RKO
RKO

Reputation: 122

You could try this approach:

def ignore_accuracy_of_class(class_to_ignore=0):
    def ignore_acc(y_true, y_pred):
        y_true_class = K.argmax(y_true, axis=-1)
        y_pred_class = K.argmax(y_pred, axis=-1)

        ignore_mask = K.cast(K.not_equal(y_pred_class, class_to_ignore), 'int32')
        matches = K.cast(K.equal(y_true_class, y_pred_class), 'int32') * ignore_mask
        accuracy = K.sum(matches) / K.maximum(K.sum(ignore_mask), 1)
        return accuracy

    return ignore_acc

Upvotes: 1

Related Questions