CODEWITHSUNDEEP
rr-caretroc
BSnider
BSnider

Reputation: 123

Selecting a Different ROC Set Point in Caret

Is it possible to select a different ROC set point in the Caret Train function instead of using metric = ROC (which I believe maximizes the AUC).

For example:

random.forest.orig <- train(pass ~ x+y, 
                 data = meter.train,
                 method = "rf",
                 tuneGrid = tune.grid,
                 metric = "ROC",
                 trControl = train.control)

Specifically I have a two class problem (fail or pass) and I want to maximize the fail predictions while still maintaining a fail accuracy (or negative prediction value) of >80%. ie for every 10 fails I predict at least 8 of them are correct.

Upvotes: 0

Views: 373

Answers (1)

Lavande
Lavande

Reputation: 894

You can customize the caret::trainControl() object to use AUC, instead of accuracy, to tune the parameters of your models. Please check the caret documentation for details. (The built-in function, twoClassSummary, will compute the sensitivity, specificity and area under the ROC curve).

Note: In order to compute class probabilities, the pass feature must be Factor

Here under is an example of using 5-fold CV:

fitControl <- caret::trainControl(
  method = "cv",
  number = 5,
  summaryFunction = twoClassSummary,
  classProbs = TRUE,
  verboseIter = TRUE
)

So your code will be adjusted a bit:

random.forest.orig <- train(pass ~ x+y, 
             data = meter.train,
             method = "rf",
             tuneGrid = tune.grid,
             metric = "ROC",
             trControl = fitControl)

# Print model to console to examine the output
random.forest.orig

Upvotes: 0

Related Questions