CODEWITHSUNDEEP
pythonmachine-learningkerasneural-network
John
John

Reputation: 643

Use validation set to determine number epochs in Keras

I was wondering whether it would be possible to use either cross validation or a fixed predefined validation set to determine the 'optimal' number of epochs for a NN using Keras in Python. I currently have the following code:

from tensorflow.keras.layers import Dense, Activation
from tensorflow.keras.models import Sequential
import pandas as pd
import numpy as np

model = Sequential()
# Adding the input layer and the first hidden layer
model.add(Dense(16, activation = 'relu', input_dim = 243))
# Adding the output layer
model.add(Dense(units = 1))
model.compile(optimizer = 'adam',loss = 'mean_squared_error')
model.fit(X, Y, batch_size = 32, epochs = 20)

If it is possible, what would I have to add to the code? I realize it is not a replicable example, though I don't think it's necessary for my question.

Upvotes: 1

Views: 1046

Answers (2)

akhetos
akhetos

Reputation: 706

You're totaly right! The best way to find "optimal" number of epoch is to use callback

That'll allow you to stop the training when you have reached the best score on your metric of interest. Here’s an example

model = Sequential()
# Adding the input layer and the first hidden layer
model.add(Dense(16, activation = 'relu', input_dim = 243))
# Adding the output layer
model.add(Dense(units = 1))
model.compile(optimizer = 'adam',loss = 'mean_squared_error')

es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=40) # Model stop training after 40 epoch where validation loss didnt decrease
mc = ModelCheckpoint('best_model.h5', monitor='val_loss', mode='min', verbose=1, save_best_only=True) #You save model weight at the epoch where validation loss is minimal
train = model.fit((train_X, train_label, batch_size=batch_size),epochs=1000,verbose=1,validation_data=(valid_X, valid_label),callbacks=[es,mc])#you can run for 1000 epoch btw model will stop after 40 epoch without better validation loss

If you want to load your weight, I'll let you check on the internet how to do it. Saving weight allows you to re-use the model without having to train it again.

For the latter, note that following validation loss score is not always the best thing to do. One exercice for you could be to find how to save model when accuracy is at its maximum.

Upvotes: 3

Arash Heidari
Arash Heidari

Reputation: 91

I don't know that I understand your question completely or not, but if you want to add sth to your code, that decides how many epochs is enough to train your data, that's not possible. You should choose the number of epochs explicitly. But other than that, you can use callbacks or TensorBoard. For example, choose your epochs to be 1000. Then you can use callbacks in a way that if the accuracy did not change stop the training. In general with callbacks, you can specify a condition, and when that condition occurs, do something else. (In our example if accuracy did not change after one epoch, stop the training). Other than that you can use TensorBoard. With this tool, you can see the history of anything related to your model after the training. Here is the link for callbacks and TensorBoard documentation: https://keras.io/callbacks/

Upvotes: 0

Related Questions