CODEWITHSUNDEEP
pythonfeature-selection
Juan Carlos
Juan Carlos

Reputation: 227

Feature Importance with SVR

I would like to plot Feature Importance with SVR, but I don't know if possible with support vector regression it's my code.

from sklearn.svm import SVR
C=1e3
svr_lin = SVR(kernel="linear", C=C)
y_lin = svr_lin.fit(X,Y).predict(X)
scores = cross_val_score(svr_lin, X, Y, cv = 5)
print(scores)
print(scores.mean())
print(scores.std())

Upvotes: 5

Views: 4977

Answers (1)

Maximus
Maximus

Reputation: 36

SVR does not support native feature importance scores, you might need to try Permutation feature importance which is a technique for calculating relative importance scores that is independent of the model used. First, a model is fit on the dataset, such as a model that does not support native feature importance scores. Then the model is used to make predictions on a dataset, although the values of a feature (column) in the dataset are scrambled. This is repeated for each feature in the dataset. Then this whole process is repeated 3, 5, 10 or more times. The result is a mean importance score for each input feature (and distribution of scores given the repeats). This approach can be used for regression or classification and requires that a performance metric be chosen as the basis of the importance score, such as the mean squared error for regression and accuracy for classification. Permutation feature selection can be used via the permutation_importance() function that takes a fit model, a dataset (train or test dataset is fine), and a scoring function.

model = SVR()
# fit the model
model.fit(X, y)
# perform permutation importance
results = permutation_importance(model, X, y, scoring='neg_mean_squared_error')
# get importance
importance = results.importances_mean
# summarize feature importance
for i,v in enumerate(importance):
    print('Feature: %0d, Score: %.5f' % (i,v))
# plot feature importance
pyplot.bar([x for x in range(len(importance))], importance)
pyplot.show()

Upvotes: 2

Related Questions