Using pROC to make a ROC curve from existing data

Question

I've already created a classifier and have run 3-fold cross-validation on the model, regenerated with several random seeds. I used thresholds at intervals of 0.1 and have already calculated the sensitivity and specificity for each threshold. My data from one seed looks like this:

Index  Seed Threshold           Fold        Sn       1-Sp
1         0       0.0              1 1.0000000 1.00000000
2         0       0.0              2 1.0000000 1.00000000
3         0       0.0              3 1.0000000 1.00000000
4         0       0.1              1 1.0000000 1.00000000
5         0       0.1              2 1.0000000 1.00000000
6         0       0.1              3 1.0000000 1.00000000
7         0       0.2              1 1.0000000 1.00000000
8         0       0.2              2 1.0000000 1.00000000
9         0       0.2              3 1.0000000 1.00000000
10        0       0.3              1 1.0000000 1.00000000
11        0       0.3              2 1.0000000 1.00000000
12        0       0.3              3 1.0000000 1.00000000
13        0       0.4              1 1.0000000 0.97435897
14        0       0.4              2 1.0000000 1.00000000
15        0       0.4              3 1.0000000 1.00000000
16        0       0.5              1 0.9523810 0.89743590
17        0       0.5              2 0.9523810 1.00000000
18        0       0.5              3 1.0000000 0.89743590
19        0       0.6              1 0.9523810 0.79487180
20        0       0.6              2 0.9047619 0.67500000
21        0       0.6              3 1.0000000 0.58974359
22        0       0.7              1 0.8571429 0.56410256
23        0       0.7              2 0.8095238 0.35000000
24        0       0.7              3 0.9523810 0.17948718
25        0       0.8              1 0.8571429 0.12820513
26        0       0.8              2 0.7142857 0.20000000
27        0       0.8              3 0.8571429 0.02564103
28        0       0.9              1 0.8571429 0.10256410
29        0       0.9              2 0.6666667 0.07500000
30        0       0.9              3 0.7619048 0.02564103
31        0       1.0              1 0.8571429 0.10256410
32        0       1.0              2 0.6666667 0.05000000
33        0       1.0              3 0.7619048 0.02564103

I was thinking I would use pROC with R to generate a ROC curve from the data and calculate the AUC. I've been looking at a lot of examples and haven't found one where I can ask the roc procedure to use data that has already been calculated. Is it possible?

I am open to other solutions/packages in R or Python if anyone has suggestions.

Answer 1

I am honestly not very familiar with ROC analysis, so your data is not very clear to me. However, we can replicate the plot that pROC::plot.roc produces using base R graphics functionalities. You may want to look into the method pROC:::plot.roc.roc to understand what it does.

Is use example data that come with the pROC package.

library(pROC)
data(aSAH)
## calculate ROC
rr <- roc(aSAH$outcome, aSAH$s100b, ci=TRUE, plot=FALSE)

## plot ROC curve with package function
plot.roc(rr)

To replicate the plot we calculate as x-values 1 - specificities, invert the x-axis, use sensitivities for the y-axis and draw a line with x=c(-1, 0), y=c(0, 1) coordinates.

with(rr, plot(-specificities, sensitivities, type="l", xaxt="n",
              xlab="", ylab="", main="My ROC curve", lwd=2,
              xlim=c(0, 1), ylim=c(0, 1)))
axis(1, axTicks(1), labels=F)
mtext(-(axTicks(1)), 1, 2, at=axTicks(1))
mtext("Specifycity", 1, 3)
mtext("Sensitivity", 2, 3)
lines(c(-1, 0), c(0, 1), col="grey")

Looks identical to me.