ROC curves from confusion matrix in MATLAB

Question

I am trying to find the best predictor for a binary outcome.

For each case, A < X and B > Y% for C = Z%. A and B are variables that are linked (Dose X to a volume Y%). C is how often this is seen in each case.

I then have different thresholds of C which I can use to predict a binary outcome (P). I also have (O) - the true outcome (binary).

I am looking at multiple X, Y and C values which best match P to O.

So for each combination of X (4 discrete points) and Y (10-90% in 10% intervals) I have a result C (%). For different thresholds of C (10-90% in 10% intervals) I have the number of cases correctly predicted and also the 2x2 confusion matrix and the sensitivity and (1-specificity).

In terms of statistics I think I can use ROC curves to find the best predictors? But I'm not sure if that's true or if I can simply compare all the combinations or just different thresholds of C of each X, Y? Or the different X,Y for the same threshold of C? Or if I should be doing a different statistical test?

But assuming I'm doing ROC curves. I plotted all the points in MATLAB (scatter) and the line y=x (refline = (1,0)). So I know the points that matter are the ones above the diagonal but how would I then fit the actual ROC curve to calculate the AUC?

I know this is confusing so I hope it makes sense!

edit: What I'm currently thinking is I need an ROC curve for the different thresholds of C for each X, Y combination. In each of these cases the best C threshold is the point nearest 1,0 (how do I find that? nearest geometrically?). And then I compare the AUC for each X,Y combination and the one with the largest area is the best?

Answer 1

I have answered similar question on these links: ROC curve and libsvm and MATLAB - generate confusion matrix from classifier
Please, go through it and let me know your doubts.