CODEWITHSUNDEEP
pythonhyperparametersxgbregressor
dado
dado

Reputation: 77

searching for best hyper parameters of XGBRegressor using HalvingGridSearchCV

I am searching for best hyper parameters of XGBRegressor using HalvingGridSearchCV. Here is the code:

base_estimator = XGBRegressor(seed=1234,use_label_encoder=False,base_score=0.5,max_delta_step=0,
                              scale_pos_weight=1,nthread=12)

params = {'learning_rate': [0.2],
        'max_depth': [500],
         'min_child_weight': [50],
         'gamma': [1.5],
         'reg_alpha': [0.7],
        'reg_lambda':[50],
        'subsample':[1],
        'colsample_bytree': [0.5],
         'n_estimators':[1000]}

sh = HalvingGridSearchCV(base_estimator, param_grid=params, cv=5,
                         factor=2, max_resources=7926,resource='n_samples',
                         aggressive_elimination=True).fit(x_train, y_train,early_stopping_rounds=10,eval_metric='rmse', eval_set=[(x_test, y_test)], verbose=True)
     
print("Best: %f using %s" % (sh.best_score_, sh.best_params_))

Best: 0.058512 using {'colsample_bytree': 0.5, 'gamma': 1.5, 'learning_rate': 0.2, 'max_depth': 500, 'min_child_weight': 50, 'n_estimators': 1000, 'reg_alpha': 0.7, 'reg_lambda': 50, 'subsample': 1}

I've tried many options, but the result is still small, Can you help me and tell me why? Is anything bad in code?

Upvotes: 0

Views: 826

Answers (2)

Yustina Ivanova
Yustina Ivanova

Reputation: 150

Can you include more parameters for search? I can see that you use the same parameters over each step of iteration. You should add some more values to increase the probability to find the ones which will be "the best". For instance, you can use this code:

params={"learning_rate": [0.1,0.2, 0.3, 0.4, 0.5, 0.8, 1,10, 20 ,0.01, 0.001, 0.0001 ],
                            "max_depth": [ 3, 4, 5, 6, 8],
                            "min_child_weight": [ 1, 3, 5, 7],
                            "gamma": [0.1,0.2, 0.3, 0.4, 0.5, 0.8, 1,10, 20 ,0.01, 0.001, 0.0001 ],
                            "colsample_bytree":[ 0.3, 0.4],
                           "early_stopping_rounds": [3]},

Upvotes: 1

HasanArcas
HasanArcas

Reputation: 37

It really depends on the data. If you have tried many hyperparameters combinations and still you have bad performance, you should try to train a model with better data.

Upvotes: 0

Related Questions