CODEWITHSUNDEEP
pythonscikit-learn
YohanRoth
YohanRoth

Reputation: 3263

How to select only valid parameters in scikit learn LinearSVC for RandomizedSearchCV

I keep suffering with my program failing due to different invalid combinations of hyperparams for LinearSVC in sklearn. Documentation does not specify in details what hyperparams work together and which ones are not. I am doing random search for hyperparams to optimize them but the function keep failing with errors about incompatible combinations.

How would I fix it?

    losses = ["hinge", "squared_hinge"]
    duals = [False, True]
    learning_rates = [1e-15, 1e-8, 1e-4, 1e-2, 1e-1, 1]
    penalties = ["l1", "l2"]
    max_iters = [1000, 5000, 10000, 20000, 50000, 150000]


    random_grid = {
        "C": learning_rates,
        "penalty": penalties,
        "max_iter": max_iters,
        "loss": losses,
        "dual": duals
    }

    svc = LinearSVC()

    n_iter = 50
    svc_random = RandomizedSearchCV(estimator=svc, param_distributions=random_grid, n_iter=n_iter,
                                    cv=3, verbose=0, n_jobs=-1)
    svc_random.fit(X, Y)
    best_params = svc_random.best_params_

Example of errors

ValueError: Unsupported set of arguments: The combination of penalty='l2' and loss='hinge' are not supported when dual=False, Parameters: penalty='l2', loss='hinge', dual=False

ValueError: Unsupported set of arguments: The combination of penalty='l1' and loss='hinge' is not supported, Parameters: penalty='l1', loss='hinge', dual=True

If I implemented this myself I could catch exceptions and proceed. But since here I am calling RandomizedSearchCV and it fails inside of it I cannot do anything... The reason I am not implementing it myself is that RandomizedSearchCV is nicely parallelized.

What are my options?

Upvotes: 1

Views: 624

Answers (1)

YohanRoth
YohanRoth

Reputation: 3263

I think I found the solution, setting error_score=np.NINF in arguments of RandomizedSearchCV seems to do the trick!

Upvotes: 1

Related Questions