CODEWITHSUNDEEP
rrandom-forest
Ayush Raj Singh
Ayush Raj Singh

Reputation: 873

Random forest package in R shows error during prediction() if there are new factor levels present in test data. Is there any way to avoid this error?

I have 30 factor levels of a predictor in my training data. I again have 30 factor levels of the same predictor in my test data but some levels are different. And randomForest does not predict unless the levels are same exactly. It shows error. Says, Error in predict.randomForest(model,test) New factor levels not present in the training data

Upvotes: 4

Views: 10346

Answers (4)

Mahi
Mahi

Reputation: 11

Simple solution to this would be

  • rbind your test data with training data
  • do prediction
  • subset the rbind data from predictions

Upvotes: 1

Rijin
Rijin

Reputation: 91

This is the issue that occurs when the level of your test data doesn't match with the level of the training data.

A simple fix that will work:

  • load test data with character column as factors
  • then rbind() test data with train data
  • Now subset the test data rows from step 2 and go for the prediction

Upvotes: 0

Mr Curious
Mr Curious

Reputation: 21

Use this to make the levels match (here test and train refer to columns in the testing and training datasets)

test<-factor(test, levels=levels(train))

Upvotes: 2

Tommy Levi
Tommy Levi

Reputation: 781

One workaround I've found is to first convert the factor variables in your train and test sets into characters

test$factor <- as.character(test$factor)

Then add a column to each with a flag for test/train, i.e.

test$isTest <- rep(1,nrow(test))
train$isTest <- rep(0,nrow(train))

Then rbind them

fullSet <- rbind(test,train)

Then convert back to a factor

fullSet$factor <- as.factor(fullSet$factor)

This will ensure that both the test and train sets have the same levels. Then you can split back off:

test.new <- fullSet[fullSet$isTest==1,]
train.new <- fullSet[fullSet$isTest==0,]

and you can drop/NULL out the isTest column from each. Then you'll have sets with identical levels you can train and test on. There might be a more elegant solution, but this has worked for me in the past and you can write it into a little function if you need to repeat it often.

Upvotes: 10

Related Questions