CODEWITHSUNDEEP
pythonscikit-learngridsearchcvmlp
Ramses Alexander Coraspe
Ramses Alexander Coraspe

Reputation: 79

How to configure the parameter_space with GridSearch?

See the example below:

parameter_space = {
    'hidden_layer_sizes': [(200,100)],
    'activation': ['tanh', 'relu'],
    'solver': ['sgd', 'adam'],
    'alpha': [0.0001, 0.05],
    'learning_rate': ['constant','adaptive'],
}

clf = GridSearchCV(mlp, parameter_space, n_jobs=-1, cv=3)

The problem is to understand how my ANN is staying internally.

I want this example to use 20 neuron for the input layer, two hidden layers of 200 and 100 neurons and an output layer with 1 neuron, because my problem is binary classification.

InputLayer (20) --- Hiddenlayer 1 (200) --- Hiddenlayer2 (100) --- OutputLayer (1)

I know that in the "hidden_layer_sizes" only hidden layers will be inserted, but how do we indicate the input and output layers? And how the "GridSearchCV" will alternate the number of neurons in the hidden layers to find the best configuration?

Upvotes: 0

Views: 505

Answers (1)

jeremy_rutman
jeremy_rutman

Reputation: 5720

Set a list of configurations to try as you did, then call gridsearch passing a reference to the MLP function (with parens, mlp()) : 

from sklearn import svm
from sklearn.model_selection import GridSearchCV
from sklearn.neural_network import MLPClassifier as mlp

parameter_space = {
    'hidden_layer_sizes': [(200, 100),(100, 200),(200, 200),(100, 100)],
    'activation': ['tanh', 'relu'],
    'solver': ['sgd', 'adam'],
    'alpha': [0.0001, 0.05],
    'learning_rate': ['constant','adaptive']
}
clf = GridSearchCV(mlp(), parameter_space, scoring='precision_macro')
print("Best params:")
print(clf.best_params_)

That will go through all the (4*2*2*2*2)=64 possible combinations of params. The hidden layer sizes are either (200,100) or (100,200) or one of the other two in the list - if you want more or less fine-grained variation you can modify that list.

Your question about input and output layer sizes is a good one, I also wondered when starting with this stuff. The answer is that the size of input vectors already determines the input layer size (it has to be the same size) and likewise with output layer and output vectors, therefore scikit doesn't need these layer sizes. Too bad other ML toolkits don't do the same!

Upvotes: 1

Related Questions