CODEWITHSUNDEEP
rrandom-forest
x00
x00

Reputation: 103

R random forest by sensitivity

Is it possible to run a supervised classification random forest maximizing sensitivity (TP/(TP+FN))? As far as I know, Accuracy or Kappa are the metric. Below, an real example where both Kappa and Accuracy miss to evaluate the model as desired. As pointed in the answer and comments (@Hanjo and @Aaron), sensitivity alone is not a good metric. 

      0    1     T  
0  1213   50  1263  
1   608   63   671  
T  1821  113  1934  

> Precisao(prev_table)
[1] "accuracy(TP+TN/T)= 0.66"
[1] "precision(TP/TP+FP)= 0.558"
[1] "sensitivity(TP/TP+FN)= 0.0939"
[1] "positive= 671 0.347"
[1] "negative= 1263 0.653"
[1] "predicted positive= 113 0.0584"
[1] "predicted negative= 1821 0.942"
[1] "Total= 1934"

This real x predicted results are poor to the goal.

Upvotes: 0

Views: 3278

Answers (1)

Hanjo Odendaal
Hanjo Odendaal

Reputation: 1441

let me elaborate for you on why choosing "sensitivity" or "specificity" as the performance metric might not be a good idea, and why I say you must perhaps go for kappa (especially in unbalanced class predictions)

Imagine we have the following dataset and prediction outcomes:

x   Outcome Prediction
0.515925884 1   1
0.416949071 0   1
0.112185499 0   1
0.557334124 0   1
0.599717812 0   1
0.272965861 1   1
0.898911346 0   1
0.347428065 0   1

If the model predicted a 1 on all observations, you would have a 100% sensitivity and would falsely presume that the model was doing well. The same is true if the model predicted all outcomes as 0, which relates to 100% specificity. But does this mean the model is well tuned? Obviously not, as a simple rule of 'predicting' all outcomes as true positives will give you specificity of 100%. Now, kappa uses the following measurement of model performance:

The Kappa statistic (or value) is a metric that compares an Observed Accuracy with an Expected Accuracy (random chance). This is a much more representative measure of the performance of your model. A nice answer to explain this can be found here Stats Exchange

Upvotes: 3

Related Questions