CODEWITHSUNDEEP
pythonmachine-learningxgboosttraining-data
phoebe
phoebe

Reputation: 13

XGBoost reporting eval-auc has been declining,but train-auc has been rising,whether the result is normal?

I want to use the early_stopping_rounds of XGBoost to do non-overfit training. For this I use following code:

parameters = {'nthread': 4,'objective': 'binary:logistic','learning_rate': 0.06,'max_depth': 6,'min_child_weight': 3,
        'silent': 0,'gamma': 0,'subsample': 0.7,'colsample_bytree': 0.5,'n_estimators': 5,
        'missing': -999,'scale_pos_weight': scale_pos_weight,'seed': 4789,'eval_metric':'auc','early_stopping_rounds': 100}
X_train, X_test, y_train, y_test =train_test_split(train_feature,train_label, test_size=0.3, random_state=4789)
dtrain = xgb.DMatrix(X_train, label=y_train)
dtest = xgb.DMatrix(X_test, label=y_test)
evallist = [(dtest, 'eval'), (dtrain, 'train')]
bst = xgb.train(parameters, dtrain,num_boost_round=1500, evals=evallist)

when print intermediate results, i get log like:

[1469]  eval-auc:0.912417   train-auc:0.986104
[16:04:23] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 110 extra nodes, 0 pruned nodes, max_depth=6
[1470]  eval-auc:0.912412   train-auc:0.986118
[16:04:27] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 102 extra nodes, 0 pruned nodes, max_depth=6
[1471]  eval-auc:0.912405   train-auc:0.986129
[16:04:30] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 116 extra nodes, 0 pruned nodes, max_depth=6
[1472]  eval-auc:0.912383   train-auc:0.986143
[16:04:34] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 116 extra nodes, 0 pruned nodes, max_depth=6
[1473]  eval-auc:0.912375   train-auc:0.986159

now I am wondering this train result is right?how to detect that if my model is overfitting or not and how many rounds to choose ?

Upvotes: 1

Views: 929

Answers (1)

carrdelling
carrdelling

Reputation: 1725

As @Stepan Novikov said, the result you see is right - your model is just starting to overfit.

Regarding your second question, the way the early_stopping_rounds parameter works is by stopping the training after N rounds have passed without any improvement in eval-aug (N is early_stopping_rounds). Note that the eval-auc value may decrease in between, but as long as there is any absolute improvement in the last N rounds, the training will continue.

In your example, the round [1469] has the maximum value for eval-auc, so the training will not stop until the round [1569] (100 rounds later, as configured).

Finally, the optimum number of rounds reached should be stored in the bst variable of your example.

Upvotes: 1

Related Questions