CODEWITHSUNDEEP
rregression
awz1
awz1

Reputation: 449

Confidence intervals for Caret package

I am using the Caret package to do some regression analysis. I have looked online but have now been able to find a solution. My code is below. Is there a way to compute the confidence intervals for this model?

set.seed(123)
# Set up repeated k-fold cross-validation
train.control <- trainControl(method = "cv", number = 10,
                              predictionBounds = c(0, NA), verboseIter = TRUE)
# Train the model
step.model <- train(`likes` ~., data = reduced_Data,
                    method = "leapSeq", 
                    tuneGrid = data.frame(nvmax = 1:13),
                    trControl = train.control)


step.model$results

step.model$bestTune

summary(step.model$finalModel)

coef(step.model$finalModel, 2)

I have tried using the predict function from the base package, but had no luck

predict(step.model$finalModel, my_data, interval = "confidence")

Upvotes: 3

Views: 1354

Answers (1)

StupidWolf
StupidWolf

Reputation: 46908

What you have is a regsubset class, there are results from different subsets of terms, and you need a criteria to choose which is the best model, refit the model and do your predictions, for example:

library(caret)
train.control <- trainControl(method = "cv",number=3)
step.model <- train(mpg ~., data = mtcars,
method = "leapSeq",trControl = train.control)

Let's say we use model with max r-squared:

res = summary(step.model$finalModel)
bestm = which.max(res$adjr2)
terms_to_use = names(which(res$which[bestm,]))[-1]
terms_to_use
1] "cyl" "hp"  "wt"

Fit the final model:

final_form = as.formula(paste("mpg ~",paste(terms_to_use,collapse="+")))
fit = lm(final_form,data=mtcars)

To get the se of the fit:

predict(fit,se=TRUE)

To get 95 ci of the coefficients:

confint(fit)
                 2.5 %       97.5 %
(Intercept) 35.0915623 42.412012412
cyl         -2.0701179  0.186884238
hp          -0.0423655  0.006289293
wt          -4.6839740 -1.649972191

Upvotes: 1

Related Questions