CODEWITHSUNDEEP
pythonmachine-learningscikit-learndecision-treeadaboost
Radu Gheorghiu
Radu Gheorghiu

Reputation: 20489

Why does AdaBoost not work with DecisionTree?

I'm using sklearn 0.19.1 with DecisionTree and AdaBoost.

I have a DecisionTree classifier that works fine:

clf = tree.DecisionTreeClassifier()

train_split_perc = 10000
test_split_perc = pdf.shape[0] - train_split_perc

train_pdf_x = pdf[:train_split_perc]
train_pdf_y = YY[:train_split_perc]

test_pdf_x = pdf[-test_split_perc:]
test_pdf_y = YY[-test_split_perc:]

clf.fit(train_pdf_x, train_pdf_y)

pred2 = clf.predict(test_pdf_x)

But when trying to add AdaBoost, it throws an error on the predict function:

treeclf = tree.DecisionTreeClassifier(max_depth=3)
adaclf = AdaBoostClassifier(base_estimator=treeclf, n_estimators=500, learning_rate=0.5)

train_split_perc = 10000
test_split_perc = pdf.shape[0] - train_split_perc

train_pdf_x = pdf[:train_split_perc]
train_pdf_y = YY[:train_split_perc]

test_pdf_x = pdf[-test_split_perc:]
test_pdf_y = YY[-test_split_perc:]

adaclf.fit(train_pdf_x, train_pdf_y)

pred2 = adaclf.predict(test_pdf_x)

Specifically the error says:

ValueError: bad input shape (236821, 6)

The dataset that it seems to be pointing to is train_pdf_y because it has a shape of (236821, 6) and I don't understand why.

From even the description of the AdaBoostClassifier in the docs I can understand that the actual classifier that uses the data is the DecisionTree:

An AdaBoost 1 classifier is a meta-estimator that begins by fitting a classifier on the original dataset and then fits additional copies of the classifier on the same dataset but where the weights of incorrectly classified instances are adjusted such that subsequent classifiers focus more on difficult cases

But still I'm getting this error.

In the code examples I've found, even on sklearn's website with how to use AdaBoost and I can't understand what I'm doing wrong.

Any help is appreciated.

Upvotes: 3

Views: 421

Answers (1)

yatu
yatu

Reputation: 88226

It looks like you are trying to perform a Multi-Output classification problem, given the shape of y, otherwise it does not make sense that you are feeding and n-dimensional y to adaclf.fit(train_pdf_x, train_pdf_y).

So assuming that is the case, the problem is that indeed Scikit-Learn's DecisionTreeClassifier does support Multi-output problems, this is, y inputs with shape [n_samples, n_outputs]. However that is not the case for the AdaBoostClassifier, given that, from the documentation, the labels must be:

y : array-like of shape = [n_samples]

Upvotes: 2

Related Questions