How a decision tree is made using cross-validation approach?

Question

I'm wonder to know how a decision tree is made when we use cross-validation, in tutorial I've read cross-validation try to find best accuracy or lowest error rate, but how a decision tree is made exactly is not clear.

For example in K=10, Is it choose the best tree from 10 other trees?

Or it tries to choose all redundant edge in tree?

I'm mean I don't understand how exactly a final tree is made from 10 other tree.

Regards.

Answer 1

Cross Validation is not a method to find an optimal model but " to derive a more accurate estimate of model prediction performance".

So it is not meant to output the best possible decision tree, but you can for example evaluate different hyper parameter settings (resulting in different decision trees) against each other with a higher statistical significance.

Answer 2

I believe this is a similar question: Help Understanding Cross Validation and Decision Trees .

Cross-validation is used to get a better estimate of any performance measure you want to look at for assessing the performance of an ML algorithm. With K=10 you will redo a tree building algorithm (like ID3 for instance) 10 times on distinct different splits of the data, where each time you train the model on 9 parts and assess the performance on the remainder (validation set). It can then be shown that the average of the 10 sets will now be less biased as a performance estimate.

Suppose we do a split of a dataset in a training and validation set. The error on the training set will be too optimistic since part can be due to overfitting. The validation error will be better, but it sucks we can't use the information in the validation set to train our models, especially when we have limited data availability. You can see cross-validation as a clever way to still make use of all the data available.