CODEWITHSUNDEEP
pythonmachine-learningxgboostxgbregressor
pg7064
pg7064

Reputation: 1

How to declare parameter grid in XGBRegressor

How do I implement the param_grid and get the best hyperparameters for xgb?

regressor = xgb.XGBRegressor()
regressor.fit(X_train,y_train)

param_grid = {
    'max_depth' :[3,4,5],
    'learning_rate':[0.1, 0.01, 0.5],
    'gamma':[0,0.25,1],
    'reg_lambda':[0, 1.0, 10.0],
    'scale_pos_weight':[1,3,5]
}

optimal_params = GridSearchCV(estimator = xgb.XGBRegressor(subsample=0.9, colsample_bytree=0.5), param_grid = param_grid, verbose = 0,)
optimal_params.fit(X_train, y_train, verbose = False)

It takes forever to compile and shows this warning repeatedly:

[16:45:55] WARNING: /workspace/src/objective/regression_obj.cu:152: 
reg:linear is now deprecated in favor of reg:squarederror.

Upvotes: 0

Views: 573

Answers (1)

Ben Reiniger
Ben Reiniger

Reputation: 12582

This all looks correct.

You have 3*3*3*3*3=243 hyperparameter combinations to check, so it probably will take some time (you can estimate this by fitting one of them first and multiplying by 243, though of course some hyperparameters will affect the training time). You might consider RandomizedSearchCV instead if it is prohibitively long.

The warning is just a warning, not an error; you can silence it as it suggests, by setting your objective to reg:squarederror (or upgrading your xgboost package, since that is the current default).

Upvotes: 1

Related Questions