CODEWITHSUNDEEP
machine-learningxgboostfeature-selectionfeature-engineering
d.c
d.c

Reputation: 77

XGBoost features with more feature importance giving less accuracy

I have six features for my model f1,f2,f3,f4,f5 and f6. And feature importance scores are in order

f1>f2>f3>f4>f5>f6

but rmse of model with features f1,f4 and f5 is less than rmse of model with features f1,f2,f3,f4,f5 and f6 or model with features f1,f2,f3. Any possible reason for this?

Upvotes: 0

Views: 258

Answers (1)

CAFEBABE
CAFEBABE

Reputation: 4101

It is hard to guess without the data. However, typically this results from correlated features. So if f2==f1 it would be the case that adding f2 to a model which contains already f1 does not provide any value. However, adding an uncorrelated feature, e.g. f4 can add a lot even so f2 > f1

Upvotes: 1

Related Questions