CODEWITHSUNDEEP
pythonkerasscikit-learngrid-searchhyperparameters
Natalia Ziemba-Jankowska
Natalia Ziemba-Jankowska

Reputation: 49

Hyperparapeters optimization with grid_search in keras and flow_from_directory

I tried to optimize hyperparameters in my keras CNN made for image classification. I decided to use grid search from sklearn. I overcame the fundamental difficulty with making x and y out of keras flow_from_directory but it still doesn't work. Error in the last line ValueError: dropout is not a legal parameter

def grid_model(optimizer='adam',
                     kernel_initializer='random_uniform', 
                     dropout=0.2,
                    loss='categorical_crossentropy'):

    model = Sequential()

    model.add(Conv2D(6,(5,5),activation="relu",padding="same",
                     input_shape=(img_width, img_height, 3)))
    model.add(MaxPooling2D((2,2)))
    model.add(Dropout(dropout))

    model.add(Conv2D(16,(5,5),activation="relu"))
    model.add(MaxPooling2D((2,2)))
    model.add(Dropout(dropout))

    model.add(Flatten())

    model.add(Dense(120, activation='relu', kernel_initializer=kernel_initializer))
    model.add(Dropout(dropout))
    model.add(Dense(84, activation='relu', kernel_initializer=kernel_initializer))
    model.add(Dropout(dropout))
    model.add(Dense(10, activation='softmax'))

    model.compile(loss=loss,
                optimizer=optimizer,
                metrics=['accuracy'])
    return model

train_generator = ImageDataGenerator(rescale=1/255)

validation_generator = ImageDataGenerator(rescale=1/255)

# Retrieve images and their classes for train and validation sets
train_flow = train_generator.flow_from_directory(directory=train_data_dir, 
                                                 batch_size=batch_size, 
                                                 target_size=(img_height,img_width))

validation_flow = validation_generator.flow_from_directory(directory=validation_data_dir, 
                                                           batch_size=batch_size,
                                                           target_size=(img_height,img_width),
                                                            shuffle = False)

clf = KerasClassifier(build_fn=grid_model(), epochs=epochs, verbose=0)

param_grid = {
    'clf__optimizer':['adam', 'Nadam'],
    'clf__epochs':[100, 200],
    'clf__dropout':[0.1, 0.2, 0.5],
    'clf__kernel_initializer':['normal','uniform'],
    'clf__loss':['categorical_crossentropy', 
                     'sparse_categorical_crossentropy', 
                     'kullback_leibler_divergence']
}

pipeline = Pipeline([('clf',clf)])

(X_train, Y_train) = train_flow.next()

grid = GridSearchCV(pipeline, cv=2, param_grid=param_grid)
grid.fit(X_train, Y_train)

Upvotes: 2

Views: 2211

Answers (1)

Manoj Mohan
Manoj Mohan

Reputation: 6044

The problem is in this line: 

clf = KerasClassifier(build_fn=grid_model(), epochs=epochs, verbose=0)

change it to 

clf = KerasClassifier(build_fn=grid_model, epochs=epochs, verbose=0)

The grid_model method should not be invoked but a reference to it should be passed.

Also, in the list of losses, 'sparse_categorical_crossentropy'(integer) cannot be used because the output shape required of the model is incompatible with that of 'categorical_crossentropy'(one-hot).

Upvotes: 2

Related Questions