CODEWITHSUNDEEP
pythonmachine-learningscikit-learnclassification
renakre
renakre

Reputation: 8291

How to use calibration plots and probability distribution to improve a classification model?

I have been working on a classification problem. With different classifiers [see figure below], the AUC scores I achieve ranges between 0.79-0.80, which is not very bad. However, I am trying to improve the performance of the classifier. To get some leads on how to do this, I have generated the following visualizations using this tutorial. Extra Trees seem to be the best. But, I do not know how to move forward after this point. For example, can I inform a VotingClassifier using this figure? If so, how? I appreciate any suggestions.

enter image description here

Upvotes: 0

Views: 1022

Answers (1)

lanenok
lanenok

Reputation: 2749

ROC_AUC score is sensitive only to the order of probabilities, not to their absolute values. Literally, if you divide all your probabilities by 2, ROC_AUC score will not change.

This means, probability calibration is useless for improving AUC. You have to resort to different methods. I don't know what you tried already, the list may include

  • feature engineering
  • feature selection
  • GridSearch for optimal hyperparameters

Upvotes: 3

Related Questions