CODEWITHSUNDEEP
matlabsvm
Matt L.
Matt L.

Reputation: 1

Matlab fitcsvm Feature Coefficients

I'm running a series of SVM classifiers for a binary classification problem, and am getting very nice results as far as classification accuracy.

The next step of my analysis is to understand how the different features contribute to the classification. According to the documentation, Matlab's fitcsvm function returns a class, SVMModel, which has a field called "Beta", defined as:

Numeric vector of trained classifier coefficients from the primal linear problem. Beta has length equal to the number of predictors (i.e., size(SVMModel.X,2)).

I'm not quite sure how to interpret these values. I assume higher values represent a greater contribution of a given feature to the support vector? What do negative weights mean? Are these weights somehow analogous to beta parameters in a linear regression model?

Thanks for any help and suggestions.

----UPDATE 3/5/15----

In looking closer at the equations describing the linear SVM, I'm pretty sure Beta must correspond to w in the primal form.

The only other parameter is b, which is just the offset.

Given that, and given this explanation, it seems that taking the square or absolute value of the coefficients provides a metric of relative importance of each feature.

As I understand it, this interpretation only holds for the linear binary SVM problem.

Does that all seem reasonable to people?

Upvotes: 0

Views: 2750

Answers (1)

Delli22
Delli22

Reputation: 305

Intuitively, one can think of the absolute value of a feature weight as a measure of it's importance. However, this is not true in the general case because the weights symbolize how much a marginal change in the feature value would affect the output, which means that it is dependent on the feature's scale. For instance, if we have a feature for "age" that is measured in years, but than we change it to months, the corresponding coefficient will be divided by 12, but clearly,it doesn't mean that the age is less important now!

The solution is to scale the data (which is usually a good practice anyway).

If the data is scaled your intuition is correct and in fact, there is a feature selection method that does just that: choosing the features with the highest absolute weight. See http://jmlr.csail.mit.edu/proceedings/papers/v3/chang08a/chang08a.pdf

Note that this is correct only to linear SVM.

Upvotes: 1

Related Questions