CODEWITHSUNDEEP
rmachine-learningrandom-forest
user2902494
user2902494

Reputation: 43

Random Forest Variable Selection

I've got a random forest which currently is built on 100 different variables. I want to be able to select only the "most important" variables to build my random forest on to try and improve performance but I don't know where to start other than getting the importance from rf$importance.

My data just consists of numerical variables which have all been scaled.

Below is my RF code:

rf.2 = randomForest(x~., data=train,importance=TRUE, ntree=1501)

#train
rf_prob_train = data.frame(predict(rf.2, newdata=train, type="prob"))
rf_prob_train <-data.frame(rf_prob_train$X0)
val_rf_train<-cbind(rf_prob_train,train$x)
names(val_rf_train)<-c("Probs","x")

##Run accuracy ratio
x<-data.frame(rcorr.cens(-val_rf_train$Probs, val_rf_train$x))
rf_train_AR<-x[2,1]
rf_train_AR

#test
rf_prob_test = data.frame(predict(rf.2, test, type="prob"))
rf_prob_test <-data.frame(rf_prob_test$X0)
val_rf_test<-cbind(rf_prob_test,test$x)
names(val_rf_test)<-c("Probs","x")

##Run accuracy ratio
x<-data.frame(rcorr.cens(-val_rf_test$Probs, val_rf_test$x))
rf_test_AR<-x[2,1]
rf_test_AR

Upvotes: 2

Views: 11676

Answers (1)

SeldomSeenSlim
SeldomSeenSlim

Reputation: 841

Busy day, so I couldn't get this to you sooner. This gives you the general idea using a generic data set.

library(randomForest)
library(datasets)

head(iris)
#To make our formula for RF easier to manipulate

var.predict<-paste(names(iris)[-5],collapse="+")
rf.form <- as.formula(paste(names(iris)[5], var.predict, sep = " ~ "))

print(rf.form)
#This is our current itteration of the formula we're using in RF

iris.rf<-randomForest(rf.form,data=iris,importance=TRUE,ntree=100)

varImpPlot(iris.rf)
#Examine our Variable importance plot

to.remove<-c(which(data.frame(iris.rf$importance)$MeanDecreaseAccuracy==min(data.frame(iris.rf$importance)$MeanDecreaseAccuracy)))
#Remove the variable with the lowest decrease in Accuracy (Least relevant variable)

#Rinse, wash hands, repeat

var.predict<-paste(names(iris)[-c(5,to.remove)],collapse="+")
rf.form <- as.formula(paste(names(iris)[5], var.predict, sep = " ~ "))

iris.rf<-randomForest(rf.form,data=iris,importance=TRUE,ntree=100)

varImpPlot(iris.rf)
#Examine our Variable importance plot

to.remove<-c(to.remove, which(data.frame(iris.rf$importance)$MeanDecreaseAccuracy==min(data.frame(iris.rf$importance)$MeanDecreaseAccuracy)))

#And so on...

Upvotes: 7

Related Questions