CODEWITHSUNDEEP
pythonscikitsscikit-learn
andreSmol
andreSmol

Reputation: 1038

Python machine learning, feature selection

I am working on a classification task related to written text and I wonder how important it is to perform some kind of "feature selection" procedure in order to improve the classification results.

I am using a number of features (around 40) related to the subject, but I am not sure if all the features are really relevant or not and in which combinations. I am experementing with SVM (scikits) and LDAC (mlpy).

If a have a mix of relevant and irrelevant features, I assume I will get poor classification results. Should I perform a "feature selection procedure" before classification?

Scikits has an RFE procedure that is tree-based that is able to rank the features. Is it meaningful to rank the features with a tree-based RFE to choose the most important features and to perform the actual classification with SVM (non linear) or LDAC? Or should I implement some kind of wrapper method using the same classifier to rank the features (trying to classify with different groups of features would be very time consuming)?

Upvotes: 3

Views: 3963

Answers (2)

Dan Stowell
Dan Stowell

Reputation: 4768

Having 40 features is not too bad. Some machine-learning is impeded by irrelevant features, but many things are quite robust to them (eg naive Bayes, SVM, decision trees). You probably don't need to do feature selection unless you decide to add many more features in.

It's not a bad idea to throw away useless features, but don't waste your own mental time on trying that out unless you have a particular motivation to.

Upvotes: 0

ogrisel
ogrisel

Reputation: 40169

Just try an see if it improves the classification score as measured with cross validation. Also before trying RFE, I would try less CPU intensive schemes such as univariate chi2 feature selection.

Upvotes: 1

Related Questions