CODEWITHSUNDEEP
pythonscikit-learnxgboost
beepretty
beepretty

Reputation: 1127

How to grid search parameter for XGBoost with MultiOutputRegressor wrapper

I'm trying to build a regressor to predict from a 6D input to a 6D output using XGBoost with the MultiOutputRegressor wrapper. I'm not sure how to do the parameter search. My code looks like this:

gsc = GridSearchCV(
            estimator=xgb.XGBRegressor(),
            param_grid={"learning_rate": (0.05, 0.10, 0.15),
                        "max_depth": [ 3, 4, 5, 6, 8],
                        "min_child_weight": [ 1, 3, 5, 7],
                        "gamma":[ 0.0, 0.1, 0.2],
                        "colsample_bytree":[ 0.3, 0.4],},
            cv=3, scoring='neg_mean_squared_error', verbose=0, n_jobs=-1)
    
grid_result = MultiOutputRegressor(gsc).fit(X_train, y_train)
self.best_params = grid_result.best_params_

However, the MultiOutputRegressor does not have the .best_params_ variable. What's the right way to do this?

Upvotes: 5

Views: 14406

Answers (2)

pfas
pfas

Reputation: 96

You can try this

gsc = GridSearchCV(
            estimator=xgb.XGBRegressor(),
            param_grid={"learning_rate": (0.05, 0.10, 0.15),
                        "max_depth": [ 3, 4, 5, 6, 8],
                        "min_child_weight": [ 1, 3, 5, 7],
                        "gamma":[ 0.0, 0.1, 0.2],
                        "colsample_bytree":[ 0.3, 0.4],},
            cv=3, scoring='neg_mean_squared_error', verbose=0, n_jobs=-1)

grid_result = MultiOutputRegressor(gsc).fit(X_train, y_train)

self.best_params = grid_result.estimators_[0].best_params_  # for the first y_target estimator

Upvotes: 8

Venkatachalam
Venkatachalam

Reputation: 16966

MultiOutputRegressor have at the estimator itself and the param_grid need to changed accordingly.


from xgboost import XGBRegressor
from sklearn.model_selection import GridSearchCV
from sklearn.datasets import make_regression
from sklearn.multioutput import MultiOutputRegressor

X_train, y_train = make_regression(n_features=6, n_targets=6)

gsc = GridSearchCV(
            estimator=MultiOutputRegressor(XGBRegressor()),
            param_grid={"estimator__learning_rate": (0.05, 0.10, 0.15)},
            cv=3, scoring='neg_mean_squared_error', verbose=0, n_jobs=-1)

grid_result = gsc.fit(X_train, y_train)
print(grid_result.best_params_)

# {'estimator__learning_rate': 0.1}

Upvotes: 1

Related Questions