CODEWITHSUNDEEP
machine-learningfeature-selection
Pikachu Sir
Pikachu Sir

Reputation: 43

What is the feature selection priciple for sequential backward selection?

For example when we do feature selection for feature A, B, C by sequential backward selection algorithm, the performance of {A, B, C} is 0.7, the performance of {A, B} is 0.9, the performance of {A} is 0.6, which feature subset will be chosen?

Upvotes: 0

Views: 205

Answers (1)

Bluexm
Bluexm

Reputation: 157

A sequential (or stepwise) backward (or forward) feature selection algorithm is usually greedy. Features will be added (respectively removed) until it doesn't improve significantly (resp. preserves) the model score.

Note that the score is not the raw performance but some other metric (AIC, BIC, adjusted R2 for example in the case of regression).

In your example, if the stopping criteria is that you want to increase performance for at lest 0.1, then the set of features {A,B} would be chosen, because at the next step the performance is down 0.3.

This wikipedia page on stepwise regression gives details in the case of regression.

Upvotes: 0

Related Questions