CODEWITHSUNDEEP
machine-learningkerasclassificationmultilabel-classification
fpi
fpi

Reputation: 323

Multi-label classification Keras metrics

Which metrics is better for multi-label classification in Keras: accuracy or categorical_accuracy? Obviously the last activation function is sigmoid and as loss function is binary_crossentropy in this case.

Upvotes: 7

Views: 12317

Answers (2)

Gianmario Spacagna
Gianmario Spacagna

Reputation: 1300

I would not use Accuracy for classification tasks with unbalanced classes. Especially for multi-label tasks, you probably have most of your labels to be False. That is, each data point can only have a small set of labels compared to the cardinality of all of the possibile labels. For that reason accuracy is not a good metric, if your model predict all False (sigmoid activation output < 0.5) then you would measure a very high accuracy.

I would analyze either the AUC or recall/precision at each epoch. Alternatively a multi-label task can be seen as a ranking task (like Recommender Systems) and you could evaluate precision@k or recall@k where k are the top predicted labels.

If your Keras back-end is TensorFlow, check out the full list of supported metrics here: https://www.tensorflow.org/api_docs/python/tf/keras/metrics.

Upvotes: 7

today
today

Reputation: 33410

Actually, there is no metric named accuracy in Keras. When you set metrics=['accuray'] in Keras, the correct accuracy metric will be inferred automatically based on the loss function used. As a result, since you have used binary_crossentropy as the loss function, the binary_accuracy will be chosen as the metric.

Now, you should definitely choose binary_accuracy over categorical_accuracy in a multi-label classification task since classes are independent from each other and the prediction for each class should be considered independently of the predictions for other classes.

Upvotes: 6

Related Questions