CODEWITHSUNDEEP
pythontensorflowkerasscikit-learndeep-learning
Ailsor
Ailsor

Reputation: 48

Grid Search fit not accepting list of tensors

I have a siamese network and I want to perform a grid seach on it using GridSearchCV.

So I create a model using the following function:

def createMod(learn_rate=0.01, optimizer='Adam'):
#K.clear_session()
# network definition
base_network = create_base_network(input_shape)

input_a = Input(shape=input_shape)
input_b = Input(shape=input_shape)

# because we re-use the same instance `base_network`,
# the weights of the network will be shared across the two branches
processed_a = base_network(input_a)
processed_b = base_network(input_b)

distance = Lambda(euclidean_distance,
                  output_shape=eucl_dist_output_shape)([processed_a, processed_b])

prediction = Dense(1,activation='sigmoid')(distance)

model = Model([input_a, input_b], prediction)

if(optimizer=='SGD'):
    opt = SGD(lr=learn_rate)
elif (optimizer=='RMSprop'):
    opt = RMSprop(lr=learn_rate)
else:
    opt = Adam(lr=learn_rate)


model.compile(loss='binary_crossentropy', optimizer=opt, metrics=[accuracy])

return model

And then I create the KerasClassifier and GridSearch as follows:

model = KerasClassifier(build_fn=createMod, verbose=0)
param_grid = dict(epochs=epochs, batch_size=batch_size, learn_rate=learn_rate,optimizer=optimizer)
grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=-1, cv=5)
X = [tr_pairs[:, 0], tr_pairs[:, 1]]
grid_result = grid.fit(X, tr_y)

However this throws the following value error:

ValueError: Found input variables with inconsistent numbers of samples: [2, 1054]

The shape of tr_pairs[:, 0] and tr_pairs[:, 1] is (1054, 6) and tr_y is (1054,)

The code for the base networks is:

def create_base_network(input_shape):
    K.clear_session()
    encoder = build_encoder(latent_dim, n_in)
    decoder = build_decoder(latent_dim, n_in)

    item = Input(shape=(n_in, ))
    encoded_repr = encoder(item)
    reconstructed_item = decoder(encoded_repr)

    autoencoder = Model(item, reconstructed_item)
    
    return autoencoder

The code for the encoder and decoder are:

def build_encoder(latent_dim, input_dim):
    input_layer = Input(shape=(input_dim, ))
    h = Dense(32, activation='relu', activity_regularizer=regularizers.l1(10e-5))(input_layer)
    h = Dropout(0.1)(h)
    h = Dense(64, activation='relu')(h)
    h = Dropout(0.1)(h)
#     h = Dense(128, activation='relu')(h)
#     h = Dropout(0.1)(h)
    latent_repr = Dense(latent_dim, activation='relu')(h)
    return Model(input_layer, latent_repr)

def build_decoder(latent_dim, input_dim):
    model = Sequential()
#     model.add(Dense(128, input_dim=latent_dim, activation='relu'))
#     model.add(Dropout(0.1))
    model.add(Dense(64, activation='relu'))
    model.add(Dropout(0.1))
    model.add(Dense(32, activation='relu'))
    model.add(Dropout(0.1))
    model.add(Dense(input_dim, activation='relu'))
    z = Input(shape=(latent_dim,))
    item = model(z)
    return Model(z, item)

The code works when I do the normal keras model's .fit function but doesn't work here.. is there a problem somewhere in my code or is it just not possible to feed multiple inputs in Grid Search and if that is the case is there a way I can still perform the grid search?

Upvotes: 1

Views: 958

Answers (1)

Marco Cerliani
Marco Cerliani

Reputation: 22031

this is workaround to pass multiple input. I create a dummy model that receives a SINGLE input in the format (n_sample, 2, 6) and then split it into two parts using Lambda layer. you can modify this according to your siamese structure.

def createMod(optimizer='Adam'):
    
    combi_input = Input((2,6)) # (n_sample, 2, 6)
    input_a = Lambda(lambda x: x[:,0])(combi_input) # (n_sample, 6)
    input_b = Lambda(lambda x: x[:,1])(combi_input) # (n_sample, 6)

    c = Concatenate()([input_a,input_b])
    x = Dense(32)(c)

    prediction = Dense(1,activation='sigmoid')(x)
    model = Model(combi_input, prediction)

    model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics='accuracy')

    return model

tr_pairs = np.random.uniform(0,1, (1054, 2, 6))
tr_y = np.random.randint(0,2, 1054)

model = tf.keras.wrappers.scikit_learn.KerasClassifier(build_fn=createMod, verbose=0)
batch_size = [10, 20]
epochs = [10, 5]
optimizer = ['adam','SGD']
param_grid = dict(batch_size=batch_size, epochs=epochs)
grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=-1, cv=3)
grid_result = grid.fit(tr_pairs, tr_y)

Upvotes: 3

Related Questions