CODEWITHSUNDEEP
tensorflowmachine-learningdeep-learningcomputer-visionpytorch
Fractale
Fractale

Reputation: 1654

Best loss function for multi-class classification when the dataset is imbalance?

I'm currently using the Cross Entropy Loss function but with the imbalance data-set the performance is not great.

Is there better lost function?

Upvotes: 2

Views: 4157

Answers (1)

Shai
Shai

Reputation: 114886

It's a very broad subject, but IMHO, you should try focal loss: It was introduced by Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He and Piotr Dollar to handle imbalance prediction in object detection. Since introduced it was also used in the context of segmentation.
The idea of the focal loss is to reduce both loss and gradient for correct (or almost correct) prediction while emphasizing the gradient of errors.

As you can see in the graph:
enter image description here

Blue curve is the regular cross entropy loss: it has on the one hand non-negligible loss and gradient even for well classified examples, and on the other hand it has weaker gradient for the erroneously classified examples.
In contrast, focal loss (all other curves) has smaller loss and weaker gradient for the well classified examples and stronger gradients for the erroneously classified examples.

Upvotes: 11

Related Questions