CODEWITHSUNDEEP
pythonrandom-forestxgboostmultilabel-classification
Nathan Chan
Nathan Chan

Reputation: 157

XGBoost hyperparameter tunning with RandomizedSearchCV with multiple classes

I am doing a hyperparameter tunning for my model and my code is like this:

para_tunning = {
        'learning_rate': [0.01,0.05,0.1],
        'min_child_weight': [1, 5, 10],
        'gamma': [0.5, 1, 1.5, 2, 5],
        'subsample': [0.6, 0.8, 1.0],
        'colsample_bytree': [0.6, 0.8, 1.0],
        'max_depth': [3, 4, 5, 6, 7, 8, 9, 10],
        "n_estimators": [100, 200, 300, 400, 500],
        "objective": "multi:softmax",
        "aplha":[0,2,4,6,8]
        }

clf_rndcv = RandomizedSearchCV(clf, 
                         param_distributions = para_tunning,
                         cv = 5,  
                         n_iter = 5,
                         scoring = 'accuracy', 
                         error_score = 0, 
                         verbose = 3, 
                         n_jobs = -1,
                         random_state = 42)
clf_rndcv.fit(X_train, y_train)

It shows that Fitting 5 folds for each of 5 candidates, totalling 25 fits, I suppose it just randomly pick 5 from the para_tunning dict and do a 5 fold cv? If I want to test all of the parameter do I switch over to gridsearchcv? And any suggestion for the tunning? I am doing a multiple class classifier with 100 classes, 500 sample per classes: so 50000 in total. Thanks!

Upvotes: 3

Views: 1934

Answers (1)

Minh-Long Luu
Minh-Long Luu

Reputation: 2751

Yes, if you want to search for ALL the hyperparameters you have to use GridSearchCV. GridSearch searches all possible combinations of the hyperparameters (it can be quite large given your case).

For your information, XGBoost has it own hyperparameters tuning. XGBoost CV

Upvotes: 1

Related Questions