CODEWITHSUNDEEP
roc
user24318
user24318

Reputation: 485

create ROC from 10 different thresholds

I have output from svmlight which has x=predictions (0.1,-0.6,1.2, -0.7...), y=actual class {+1,-1}. I want to create an ROC curve for 10 specific different thresholds (let t be a vector that contains 10 different threshold values). I checked ROCR package but I didn't see any option for supplying threshold vector. I need to calculate TPR and FPR for each threshold value and plot. Is there any other way to do that ? I am new to R programming.

Upvotes: 1

Views: 2540

Answers (3)

Asya Lyanova
Asya Lyanova

Reputation: 49

You can use this func:

def roc_curve_new(y_true, y_pred, thresholds):
    fpr_list = []
    tpr_list = []
    thresholds_list = []
    for threshold in thresholds:
        thresholds_list.append(threshold)
        new_y_pred = np.where(y_pred < threshold, y_pred, 1)
        y_pred_b = np.where(new_y_pred >= threshold,new_y_pred, 0)
    
        tn, fp, fn, tp = confusion_matrix(list(y_true), list(y_pred_b)).ravel()
        #true positive rate
        tpr = tp/(tp+fn)
        #false positive rate
        fpr = fp/(fp+tn)
        fpr_list.append(fpr)
        tpr_list.append(tpr)
return fpr_list, tpr_list, thresholds_list

thresholds = np.arange(0.1, 1.1, 0.1)
y = np.array([1, 1, 0, 1, 1, 0, 0])
scores = np.array([0.5, 0.4, 0.35, 0.75, 0.55, 0.4, 0.2])
fpr, tpr, _ = roc_curve_new(y, scores, thresholds)
plt.plot(fpr, tpr, '.-', color='b')
plt.plot([0, 1], [0, 1], color="navy", lw=1, linestyle="--")  
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel("False Positive Rate")
plt.ylabel("True Positive Rate")

It will give you img:ROC-curve

Upvotes: 0

Calimo
Calimo

Reputation: 7959

You can do that with the pROC package. First create the ROC curve (for all thresholds):

myROC <- roc(y, x) # with the x and y you defined in your question

And then you query this curve for the 10 (or any number of) thresholds that you stored in t:

coords(myROC, x = t, input="threshold", ret = c("threshold", "se", "1-sp"))

Sensitivity is your TPR while 1-Specificity is your FPR.

Disclaimer: I am the author of pROC.

Upvotes: 2

patr1ckm
patr1ckm

Reputation: 1196

ROCR creates an ROC curve by plotting the TPR and FPR for many different thresholds. This can be done with just one set of predictions and labels because if an observation is classified as positive for one threshold, it will also be classified as positive at a lower threshold. I found this paper to be helpful in explaining ROC curves in more detail.

You can create the plot as follows in ROCR where x is the vector of predictions, and y is the vector of class labels:

pred <- prediction(x,y) 
perf <- performance(pred,"tpr","fpr")
plot(perf)

If you want to access the TPR and FPR associated with all the thresholds, you can examine the performance object 'perf':

str(perf)

The following answer shows how to obtain the threshold values in more detail:

https://stackoverflow.com/a/16347508/786220

Upvotes: 2

Related Questions