CODEWITHSUNDEEP
pythontensorflowmachine-learningkerasscikit-learn
Xu.Haijiang
Xu.Haijiang

Reputation: 21

Why did RandomizedSearchCV raise the error when I use it to search parameters in TF 2.0?

My code is as such, I try to search parameters in TF 2.0 used the KerasRegressor API.

def build_model(hidden_layers = 1,
                layer_size = 30,
                learning_rate = 3e-3):
    model = keras.models.Sequential()
    model.add(keras.layers.Dense(layer_size, activation='relu',
                                 input_shape=x_train.shape[1:]))
    for _ in range(hidden_layers - 1):
        model.add(keras.layers.Dense(layer_size,
                                     activation = 'relu'))
    model.add(keras.layers.Dense(1))
    optimizer = keras.optimizers.SGD(learning_rate)
    model.compile(loss = 'mse', optimizer = optimizer)
    return model

sklearn_model = tf.keras.wrappers.scikit_learn.KerasRegressor(
    build_fn = build_model)
callbacks = [keras.callbacks.EarlyStopping(patience=5, min_delta=1e-2)]
history = sklearn_model.fit(x_train_scaled, y_train,
                            epochs = 10,
                            validation_data = (x_valid_scaled, y_valid),
                            callbacks = callbacks)

from scipy.stats import reciprocal
param_distribution = {
    "hidden_layers":[1, 2, 3, 4],
    "layer_size": np.arange(1, 100),
    "learning_rate": reciprocal(1e-4, 1e-2),
}

from sklearn.model_selection import RandomizedSearchCV

random_search_cv = RandomizedSearchCV(sklearn_model,
                                      param_distribution,
                                      n_iter = 10,
                                      cv = 3,
                                      n_jobs = 1)
random_search_cv.fit(x_train_scaled, y_train, epochs = 100,
                     validation_data = (x_valid_scaled, y_valid),
                     callbacks = callbacks)

However it end up with the error:

Cannot clone object , as the constructor either does not set or modifies parameter layer_size

In my view, I have contained the parameter layer_size.

What should I do to solve this problem? also, when I change the "n_jobs" > 1, it can't work.

Upvotes: 2

Views: 450

Answers (2)

Edson Cilos
Edson Cilos

Reputation: 36

This an issue in Kera's scikit learn wrapper, which can be solved in the following way:

  1. Find your tensorflow's folder installation and the file tensorflow\python\keras\wrappers\scikit_learn.py

  2. Edit the file in the line 117:

remove: res = copy.deepcopy(self.sk_params)

add: res = self.sk_params.copy()

It basically switch from a deep copy to a shalow one.

Source here.

Upvotes: 1

Samuel Prevost
Samuel Prevost

Reputation: 1104

This is not a full answer but it seems like there is a bug in the cloning process due to the way Keras' scikit learn wrapper returns the parameters (see get_params).

An issue is opened about it on Keras' GitHub.

A workaround for flat parameters ranges is to convert them to Python list using numpy's .tolist() method.

Upvotes: 0

Related Questions