CODEWITHSUNDEEP
pythonscikit-learngrid-search
Maryam Hnr
Maryam Hnr

Reputation: 155

How to read selected builtin parameters of sklearn grid search estimators?

I'm using sklearn 'gridsearchcv' to find the best hyper-parameters of my feature selection method, here 'selectkbest'. I'm useing a pipeline of 'feature selection' and my ridge regression on top of it.

An example of such pipeline is here.

However, I'm using 'RidgeCV' which chooses it's own regularizer with an efficient builtin loocv. In this case I'm not able to access the parameters of my Ridgecv.

k_range = [10, 15, 30, 50, 150, 300, 500]
alpha_range = [0.01, 0.05, 
           0.1, 0.5, 
           1,
           2,
           5, 
            10,
            100]
featureSelection = SelectKBest(f_classif)
ridgecv = linear_model.RidgeCV(alphas=alpha_range, gcv_mode='eigen', 
                           store_cv_values=True)
pipe = Pipeline([('anova', featureSelection), ('ridgecv', ridgecv)])
grid = GridSearchCV(pipe, param_grid={'anova__k':k_range}, 
cv=inner_cv)
grid.fit(x_train, y_train)
print(grid.best_params_)

In putput of this code I can see what is the selected k for 'anova' feature selection but I can't find a way to access the chosen 'alpha' for my 'ridgecv'

{'anova__k': 15}

I try to get the best estimator and read it's alpha parameter like this:

es = grid.estimator.named_steps['ridgecv'].score(x,y)
es.alpha_

But it says:

NotFittedError: This RidgeCV instance is not fitted yet. Call 'fit' with appropriate arguments before using this method.

Upvotes: 0

Views: 652

Answers (1)

JARS
JARS

Reputation: 1119

You might want to try best_estimator_ instead of estimator, ie 

es = grid.best_estimator_.named_steps['ridgecv']

On a different topic, while strictly functional, it is strange to have cross-validation both at RidgeCV level and at GridSearchCV level. In this case, I would suggest working with the sklearn.linear_model.Ridge class and have the cross-validation of alpha at the GridSearchCV level, e.g.

ridge = Ridge()
pipe = Pipeline([('anova', featureSelection), ('ridge', ridge)])
grid = GridSearchCV(pipe, param_grid={'anova__k':k_range, 'ridge__alpha': alpha_range})

Upvotes: 1

Related Questions