CODEWITHSUNDEEP
pythondecision-treexgboostxgbclassifier
ErickV
ErickV

Reputation: 61

Xgboost - Decision Tree - Only one leaf

I'm try to plot a decision trees of Xgboost models from different datasets.

Which worked fine for most of them, but for one dataset the plot_tree just shows only one leaf.

It's weird for me, once the max_depth of that model is 5.

Could anyone give me a tip?

Thanks for considering my question. :) ! 🙏

Upvotes: 1

Views: 1261

Answers (1)

ErickV
ErickV

Reputation: 61

I am glad to share that I figured out the reason for my problem :)

XGBoost is a technique which works on the principle of ensemble, so XGBClassifier creates multiple trees and some trees can ended in only one leaf. I realised that the functions used to plot export_graphviz or plot_tree plotted the first tree of my model as default and not the best interaction. To do that I must set the parameter "num_trees":

"num_trees (int, default 0) – Specify the ordinal number of target tree"

So, I have to find the ordinal number of the target tree. Fortunately, there are two functions that set it for us .get_booster () and .best_iteration.

See below the code to plot the tree with the best interaction.

plot_tree (model, ax = ax, num_trees = model.get_booster().best_iteration)

Upvotes: 4

Related Questions