CODEWITHSUNDEEP
roptimization
Fritz
Fritz

Reputation: 57

Calculate the 'optimal' (=max sensitivity and specificity) cut-off value using ROCR

I would like to calculate the optimal cut-off value, in my case the intersection of maximum sensitivity and specificity to define a decision rule for a logistic regression classification approach. Looking for a solution in stack overflow I found a suggested solution to calculate cut-off that max sensitivity vs specificity using ROCR.

However, when I'm plotting the specificity and the sensitivity values (y-axis) on a joint scale as a function of cut-off values (x-values) of my prediction-object (calculated by the eRm package) with the ROCR package, I got the following figure (see below).

Now, if I calculate the point of intersection of both functions, where specificity and sensitivity were maximized, as suggested in the prior thread, I got a value that lies elsewhere next to the point, which I would visually detect as point of intersection.

My question is rather simple: Could someone show me a way to calculate the point of intersection of both functions to get an 'optimal' cut-off point in R?

Figure 1: Example Plot of sensitivity and specificity as a function of probability cutoff. The line indicates the 'optimal' cutoff value deviating from the visually detected optimal threshold. The calculation of the 'optimal' cutoff value has been done as suggested in an earlier stack overflow thread

library(ROCR)
library(eRm)
set.seed(1)
data <- sim.rasch(30, 300) # simulate Rasch homogenous data 
model.RM<-RM(data, se=T)#estimate Rasch model
PPAR.X <-person.parameter(model.RM)
#Goodness-of-fit test (see Mair et al. 2008)
gof.model.RM<-gofIRT(PPAR.X)
#summary(gof.model.RM) 

#ROCR
pred.model.RM <- gof.model.RM$predobj
Sens.model.RM <- performance(pred.model.RM,  measure="sens", x.measure="cutoff")
Spec.model.RM <- performance(pred.model.RM,  measure="spec", x.measure="cutoff")

#Identify the 'optimal' cutoff that yields the highest sensitivity and specificity according to prior stack overflow thread:
SensSpec.model.RM <- performance(pred.model.RM,  "sens", "spec")
CP<[email protected][[1]][which.max([email protected][[1]][email protected][[1]])]
# [1] 0.5453864 # 'optimal' cutoff value

#Plot
plot(Sens.model.RM, type="l", col="red",xlab="",ylab="")
par(new=TRUE)
plot(Spec.model.RM, type="l", col="blue", xlab="Probability cutoff (threshold)",ylab="Sensitivity/Specificity")
abline(v = CP, col = "black", lty = 3)#add a line indicating the suggested 'optimal' cutoff value differing from the visually expected one

Upvotes: 3

Views: 6486

Answers (1)

MrFlick
MrFlick

Reputation: 206308

If you want to find the largest sum, you can do

best.sum <- which.max([email protected][[1]][email protected][[1]])
[email protected][[1]][best.sum]
# [1] 0.5453863

If you want to find the closest intersection, you can do

both.eq <- which.min(abs([email protected][[1]][email protected][[1]]))
[email protected][[1]][both.eq]
# [1] 0.5380422

Upvotes: 4

Related Questions