CODEWITHSUNDEEP
python-3.xtensorflowmachine-learningkerasartificial-intelligence
hhaefliger
hhaefliger

Reputation: 531

Keras accuracy of only most certain predictions

I am using keras to perform binary classification (single label, one or zero).

I would like to create a custom metric that takes the highest 10% from the predictions and calculates the accuracy of those.

So for example with the labels [1, 0, 1, 1, 0, 1, 0, 1, 0, 1]

and the predictions [0.5, 0.7, 0.98, 0.1, 0.2, 0.5, 0.2, 0.1, 0.9, 0.8]

It only computes the accuracy of the 0.98 prediction.

Upvotes: 0

Views: 242

Answers (2)

hhaefliger
hhaefliger

Reputation: 531

I managed to solve my problem using this code:

import tensorflow as tf

def topacc(y_true, y_pred):
  k = tf.cast(len(y_true) // 10, 'int64')
  y_true, y_pred = tf.transpose(y_true), tf.transpose(y_pred)
  return tf.keras.metrics.top_k_categorical_accuracy(y_true, y_pred, k=k)

This works as a full keras metric

Upvotes: 1

Andrey
Andrey

Reputation: 6367

Try this code:

import tensorflow as tf

def accuracy_metric_fn(y_true, y_pred):
  l = len(y_pred) // 10
  top = tf.math.top_k(y_pred, l)
  y_pred = top.values
  y_true = tf.gather(y_true, top.indices)
  y_true = tf.cast(y_true, tf.float32)
  loss = tf.keras.metrics.binary_accuracy(
      y_true, y_pred)
  return loss

y_true = [1, 0, 1, 1, 0, 1, 0, 1, 0, 1]
y_pred = [0.5, 0.7, 0.98, 0.1, 0.2, 0.5, 0.2, 0.1, 0.9, 0.8]
print(accuracy_metric_fn(y_true, y_pred).numpy())

Upvotes: 0

Related Questions