CODEWITHSUNDEEP
pythondecision-treexgboost
qalis
qalis

Reputation: 1523

XGBoost - select tree with the highest accuracy

I need to visualize XGBoost's effects and I want to extract a single tree from it that has the highest accuracy on the test set. Is is built in in any way, or do I have to test by hand all of the trees? I'm using the Python version.

Upvotes: 0

Views: 174

Answers (1)

user1808924
user1808924

Reputation: 4926

If you'd like to inspect individual trees in a tree ensemble model, then you should consider switching from a boosting-type algorithm (eg. XGBoost, LightGBM, GBM) to a bagging-type algorithm (eg. Random Forest).

By definition, the bagging algorithm grows trees in decreasing "smartness" order. In this case, the highest accuracy tree model is probably the first one in the XGBoost tree ensemble.

Upvotes: 0

Related Questions