CODEWITHSUNDEEP
catboost
ajaix
ajaix

Reputation: 61

skopt's BayesSearchCV with CatBoost

I am trying to use the skopt's BayesSearchCV method with Catboost.However i am confused on where to pass in the indices of categorical features native to catboost on to bayesSearch object's fit() method.

clf = catboost.CatBoostClassifier()
search_spaces = {'iterations': (10, 1000),
                 'depth': (1, 10),
                 'learning_rate': (0.001, 0.5),
                 'random_strength': (1e-9, 10)}
pt = BayesSearchCV(clf,
                    search_spaces,
                    n_iter=40)
pt.fit(x_train,y_train)

fit() throws error couldn't convert categorical to float,

i know, the error is because i am not passing the indices of the categorical variables but i can't pass them in the bayes's fit() method.Also tried the catboost's pool() method which din't work either.

Upvotes: 2

Views: 880

Answers (1)

Jayant Sahewal
Jayant Sahewal

Reputation: 561

I had a similar problem with CalibratedClassifierCV. So, posting the solution here. Maybe that would work for you.

You can try approaching this problem by first encoding categorical features into numerical just to fool BayesSearchCV but still pass the categorical feature indices while initiating CatBoostClassifier() so CatBoost can leverage them appropriately:

# Import necessary libraries
from sklearn.calibration import CalibratedClassifierCV
from catboost import CatBoostClassifier

# Initialize Calibrated Classifier with Catboost
model_combined = CalibratedClassifierCV(
    CatBoostClassifier(cat_features=<cat_features_indexes>),
    method='isotonic',
    cv=5,
)

Upvotes: 1

Related Questions