CODEWITHSUNDEEP
pythonscikit-learnfeature-selectiongrid-searchrfe
Kate
Kate

Reputation: 11

retrieving selected features from RFECV inside gridsearchCV

I want to write a code that grid searches over a couple of processors and preprocessors, but also over different combinations of features. I did this by using RFECV inside a gridsearchCV. However, this takes ages to run. Therefor, I reversed the order. I did a grid search and then I put it inside RFECV. Now, I want to see and print which features are actually selected in the best model. I tried a couple of solutions from this website, but none of them have worked. How can I access the selected features? Both grid_dem.get_support(indices=True) and grid_dem.support_ did not work. I get this and other similar errors: AttributeError: 'RFECV' object has no attribute 'support_'

The relevant part of my code is:

pipe = Pipeline([('preprocessing', StandardScaler()), ('rbf_svm', SVC(max_iter=1e6))])
param_grid ={'preprocessing': [StandardScaler(), MinMaxScaler(), Normalizer(), RobustScaler()],
     'rbf_svm__kernel': ['rbf', 'poly', 'linear', 'sigmoid'],
     'rbf_svm__C': np.logspace(-3,2,5), 'rbf_svm__gamma': np.logspace(-3,2,5)}

    # {'preprocessing': [StandardScaler(), MinMaxScaler(), Normalizer(), RobustScaler()],
    #  'rbf_svm': [LogisticRegression(max_iter=1e6)],
    #  'logisticregression__C': np.logspace(-3,2,5)}]

grid_dem = GridSearchCV(pipe, param_grid, cv=5,verbose=5,n_jobs=3)
grid_dem.fit(X_democrat_train,y_democrat_train)
grid_dem.score(X_democrat_test,y_democrat_test)
print(grid_dem.best_estimator_)
rfecv=RFECV(grid_dem, verbose=3)
print(rfecv)
print(rfecv.get_support(indices=True))
# rfecv=rfecv.fit_transform(X_democrat_train, y_democrat_train)
# print(rfecv.get_params())

As you can see in the last two lines, I also tried to transform the X, but this did not work either.

Upvotes: 1

Views: 1256

Answers (1)

Vivek Kumar
Vivek Kumar

Reputation: 36599

You have sent a pipeline to gridSearch. So the best_estimator_ will return a pipeline. Butgrid_dem is still a GridSearchCV object. So obviously get_support() will not work. After that you passed this grid_dem to RFECV but did not call fit() on it. Thats why the support_ is not available.

Please update your code like this:

rfecv=RFECV(grid_dem.best_estimator_, verbose=3)  <==Edited
rfecv.fit(X_democrat_train,y_democrat_train)   #<==This is what you want
print(rfecv.get_support(indices=True))  
# rfecv=rfecv.fit_transform(X_democrat_train, y_democrat_train)
# print(rfecv.get_params())mocrat_train,y_democrat_train)

Upvotes: 1

Related Questions