CODEWITHSUNDEEP
pythonmachine-learningartificial-intelligencexgboost
syllogismos
syllogismos

Reputation: 660

How to boost on an existing xgboost model from its last iteration without starting from the beginning for multi:prob

I read that it is one of the advantages of xgboost, that you can train on an existing model. Say I trained my model for 100 iterations, and want to restart from there to finish another 100 iterations, instead of redoing everything from the scratch..

I found this in xgboost demo examples, from here https://github.com/dmlc/xgboost/blob/master/demo/guide-python/evals_result.py

bst = xgb.train( param, dtrain, 1, watchlist )
ptrain = bst.predict(dtrain, output_margin=True)
ptest  = bst.predict(dtest, output_margin=True)
dtrain.set_base_margin(ptrain)
dtest.set_base_margin(ptest)
print ('this is result of running from initial prediction')
bst = xgb.train( param, dtrain, 1, watchlist )

but this particular example is for objective, binary:logistic.. if I do this, I'm getting this error on set_base_margin

TypeError: only length-1 arrays can be converted to Python scalars

I have a model that got trained for 100 iterations.. I want to do another 100 iterations, but don't want to begin from the start again.

Any help..??

Upvotes: 4

Views: 3104

Answers (2)

Ioannis Nasios
Ioannis Nasios

Reputation: 8527

Things have changed now....

bst = xgb.train(param, dtrain, 1, watchlist , xgb_model=bst )

Upvotes: 2

syllogismos
syllogismos

Reputation: 660

figured it out, from this issue in xgboost repo https://github.com/dmlc/xgboost/issues/235

Yes, this is something we overlooked when designing the interface, you should be able to set_margin with flattened array.

set_base_margin expects a 1d array, so you just need to flatten the margined predictions and then pass it to set_base_margin

in the above code add these lines before setting the base margin

ptrain = ptrain.reshape(-1, 1)
ptest = ptest.reshape(-1, 1)

and training on the new dtrain with updated base margins will continue iterating from that stage

Upvotes: 3

Related Questions