CODEWITHSUNDEEP
tensorflowmachine-learningdeep-learningkeras
MTANG
MTANG

Reputation: 516

keras resume training with different learning rate

I built a simple LSTM model using keras and trained as follows:

model = Sequential()
model.add(LSTM(activation='tanh',input_dim=6,output_dim=50,return_sequences=False))
model.add(Dense(output_dim=1,activation = 'sigmoid'))
model.compile(loss='binary_crossentropy', optimizer =optimizers.Adam(lr = 0.01),metrics=['accuracy'])
model.fit(X_train,y_train,batch_size=256,nb_epoch=1000,validation_data = (X_test,y_test))
model.save('model_params.h5')

The model almost converged. Therefore I want to fine tune the model by resuming training using smaller learning rate (i.e., 0.001). How could I achieve this?

Upvotes: 3

Views: 2186

Answers (1)

Daniel Möller
Daniel Möller

Reputation: 86600

New answer

If your optimizer has an lr property, and this property is a tensor, you can change it with:

keras.backend.set_value(model.optimizer.lr, new_value)

Old answer, with some side effects

You just need to compile the model again:

model.compile(loss='binary_crossentropy',
              optimizer= optimizers.Adam(lr=0.001),...)

But usually, Adam is a very good optimizer and doesn't need those changes. It's normal for it to find alone its ways.

It's very normal for the training to diverge when you compile with a new optimizer. It takes a few epochs until the optimizer adjusts itself.

Upvotes: 4

Related Questions