CODEWITHSUNDEEP
pythonscikit-learnrandom-forest
coffee
coffee

Reputation: 713

Random Forest implementation in Python

all!

Could anybody give me an advice on Random Forest implementation in Python? Ideally I need something that outputs as much information about the classifiers as possible, especially:

  1. which vectors from the train set are used to train each decision trees
  2. which features are selected at random in each node in each tree, which samples from the training set end up in this node, which feature(s) are selected for split and which threashold is used for split

I have found quite some implementations, the most well known one is probably from scikit, but it is not clear how to do (1) and (2) there (see this question). Other implementations seem to have the same problems, except the one from openCV, but it is in C++ (python interface does not cover all methods for Random Forests).

Does anybody know something that satisfies (1) and (2)? Alternatively, any idea how to improve scikit implementation to get the features (1) and (2)?

Solved: checked the source code of sklearn.tree._tree.Tree. It has good comments (which fully describe the tree):

 children_left : int*
    children_left[i] holds the node id of the left child of node i.
    For leaves, children_left[i] == TREE_LEAF. Otherwise,
    children_left[i] > i. This child handles the case where
    X[:, feature[i]] <= threshold[i].

children_right : int*
    children_right[i] holds the node id of the right child of node i.
    For leaves, children_right[i] == TREE_LEAF. Otherwise,
    children_right[i] > i. This child handles the case where
    X[:, feature[i]] > threshold[i].

feature : int*
    feature[i] holds the feature to split on, for the internal node i.

threshold : double*
    threshold[i] holds the threshold for the internal node i.

Upvotes: 3

Views: 4019

Answers (1)

Andreas Mueller
Andreas Mueller

Reputation: 28758

You can get nearly all the information in scikit-learn. What exactly was the problem? You can even visualize the trees using dot. I don't think you can find out which split candidates were sampled at random, but you can find out which were selected in the end. Edit: Look at the tree_ property of the decision tree. I agree, it is not very well documented. There really should be an example to visualize the leaf distributions etc. You can have a look at the visualization function to get an understanding of how to get to the properties.

Upvotes: 2

Related Questions