CODEWITHSUNDEEP
rrocauc
Bfu38
Bfu38

Reputation: 1141

AUC unexpected value

I have the following predictions after running a logistic regression model on a set of molecules we suppose that are predictive of tumors versus normals.

                  Predicted   class     
                      T        N          
                 T   29        5
  Actual class
                 N   993      912

I have a list of scores that range from predictions <0 (negative numbers) to predictions >0 (positive numbers). Then I have another column in my data.frame that indicated the labels (1== tumours and 0==normals) as predicted from the model. I tried to calculate the ROC using the library(ROC) in the following way:

 pred = prediction(prediction, labels)     
 roc = performance(pred, "tpr", "fpr")   
 plot(roc, lwd=2, colorize=TRUE)

Using:

       roc_full_data <- roc(labels, prediction)
       rounded_scores <- round(prediction, digits=1)
       roc_rounded <- roc(labels, prediction)

Call:

       roc.default(response = labels, predictor = prediction)
       Data: prediction in 917 controls (category 0) < 1022 cases (category1).
       Area under the curve: 1

The AUC is equal to 1. I'm not sure that I run all correctly or probably I'm doing something wrong in the interpretation of my results because it is quite rare that the AUC is equal to 1.

Upvotes: 0

Views: 321

Answers (2)

Fernando
Fernando

Reputation: 7895

I use pROC to calculate AUC:

require(pROC)
set.seed(1)
pred = runif(100)
y = factor(sample(0:1, 100, TRUE))
auc = as.numeric(roc(response = y, predictor = pred)$auc)
print(auc) # 0.5430757

Or

require(AUC)
auc = AUC::auc(AUC::roc(pred, y))
print(auc) # 0.4569243

I can't explain why the results are different.

EDIT: The above aucs sum to 1.0, so one of the libs automatically 'inverted' the predictions.

Upvotes: 0

Jil Jung Juk
Jil Jung Juk

Reputation: 700

There is a typo in your x.measure which should have thrown an error. You have "for" and not "fpr". Try the following code.

performance(pred, measure = "tpr", x.measure = "fpr")
plot(perf)

# add a reference line to the graph
abline(a = 0, b = 1, lwd = 2, lty = 2)

# calculate AUC
perf.auc <- performance(pred, measure = "auc")
str(perf.auc)
as.numeric([email protected])

Upvotes: 1

Related Questions