CODEWITHSUNDEEP
rglmauc
natasha
natasha

Reputation: 21

Calculating AUC of training dataset for glm function in R

I am trying to find AUC on a training data for my logistic regression model using glm

I split data to train and test set, fitted a logistic regression model regression model using glm, computed predicted value and trying to find AUC

d<-read.csv(file.choose(), header=T)
 set.seed(12345)
 train = runif(nrow(d))<.5
 table(train)
 fit = glm(y~ ., binomial, d)
 phat<-predict(fit,type = 'response')
 d$phat=phat
 g <- roc(y ~ phat, data = d, print.auc=T)
 plot(g)

Upvotes: 1

Views: 7729

Answers (2)

Joris C.
Joris C.

Reputation: 6234

Another user-friendly option is to use the caret library, which makes it pretty straightforward to fit and compare regression/classification models in R. The following example code uses the GermanCredit dataset to predict credit worthiness using a logistic regression model. The code is adapted from this blog: https://www.r-bloggers.com/evaluating-logistic-regression-models/.

library(caret)

## example from https://www.r-bloggers.com/evaluating-logistic-regression-models/
data(GermanCredit)

## 60% training / 40% test data
trainIndex <- createDataPartition(GermanCredit$Class, p = 0.6, list = FALSE)

GermanCreditTrain <- GermanCredit[trainIndex, ]
GermanCreditTest <- GermanCredit[-trainIndex, ]

## logistic regression based on 10-fold cross-validation 
trainControl <- trainControl(
     method = "cv",
     number = 10,
     classProbs = TRUE,
     summaryFunction = twoClassSummary
)

fit <- train(
    form = Class ~ Age + ForeignWorker + Property.RealEstate + Housing.Own + 
         CreditHistory.Critical,  
    data = GermanCreditTrain,
    trControl = trainControl,
    method = "glm", 
    family = "binomial", 
    metric = "ROC"
)

## AUC ROC for training data
print(fit)

## AUC ROC for test data
## See https://topepo.github.io/caret/measuring-performance.html#measures-for-class-probabilities
 predictTest <- data.frame(
         obs = GermanCreditTest$Class,                                    ## observed class labels
         predict(fit, newdata = GermanCreditTest, type = "prob"),         ## predicted class probabilities
         pred = predict(fit, newdata = GermanCreditTest, type = "raw")    ## predicted class labels
     ) 

twoClassSummary(data = predictTest, lev = levels(predictTest$obs))

Upvotes: 1

crich
crich

Reputation: 99

I like using the performance command found in the ROCR library.

library(ROCR)
# responsev = response variable

d.prediction<-prediction(predict(fit, type="response"), train$responsev)
d.performance<-performance(d.prediction,measure = "tpr",x.measure="fpr")
d.test.prediction<-prediction(predict(fit,newdata=d.test, type="response"), d.test$DNF)
d.test.prefermance<-performance(d.test.prediction, measure="tpr", x.measure="fpr")

# What is the actual numeric performance of our model?
performance(d.prediction,measure="auc")
performance(d.test.prediction,measure="auc")

Upvotes: 0

Related Questions