CODEWITHSUNDEEP
r
user1398057
user1398057

Reputation: 1159

GBM in R for adaBoost ~ predict() values lie outside of [0,1]

I am currently trying to fit an adaBoost model in R using the gbm.fit model. I have tried everything I could but in the end my model keeps giving me prediction values outside of [0,1]. I understand that type = "response" only works for bernoulli but I keep getting values just outside of 0,1. Any thoughts? Thanks! 

GBMODEL <- gbm.fit(
               x=training.set,
               y=training.responses,
               distribution="adaboost",
               n.trees=5000,
               interaction.depth=1,
               shrinkage=0.005,
               train.fraction=1,
               )

predictionvalues = predict(GBMODEL, 
                  newdata=test.predictors,
                  n.trees=5000,
                  type="response")

Upvotes: 4

Views: 2279

Answers (2)

William Fang
William Fang

Reputation: 46

it is correct to obtain y range outside [0,1] by gbm package choosing "adaboost" as your loss function. After training, adaboost predicts category by the sign of output.

For instance, for binary class problem, y{-1,1}, the class lable will be signed to the sign of output y. So if you got y=0.9 or y=1.9 will give you the same result-observation belongs to y=1 class. However, y=1.9 simply suggests a more confident conclusion than y=0.9. (if you want to know why, I would suggest you to read margin-based explanation of adaboost, you will find very similar result with SVM).

Hope this can help you.

Upvotes: 2

Caleb
Caleb

Reputation: 11

This may not be completely accurate mathematically, but I just did pnorm( predicted values) and you get values from 0 to 1, because the adaboost predicted values appear to be scaled on a Normal(0,1).

Upvotes: -1

Related Questions