CODEWITHSUNDEEP
machine-learningdeep-learningkeras
nabroyan
nabroyan

Reputation: 3275

Keras EarlyStopping patience parameter

I'm trying to do some binary classification and I use Keras's EarlyStopping callback. However, I have a question regarding patience parameter.

In the documentation it is stated

patience: number of epochs with no improvement after which training will be stopped.

but I find that it behaves in a different way. For example, I have set

EarlyStopping(monitor='val_loss', min_delta=0.0001, patience=2, verbose=0, mode='auto')

and here are the results:

val_loss: 0.6811
val_loss: 0.6941 
val_loss: 0.6532
val_loss: 0.6546
val_loss: 0.6534
val_loss: 0.6489
val_loss: 0.6240
val_loss: 0.6285
val_loss: 0.6144
val_loss: 0.5921
val_loss: 0.5731
val_loss: 0.5956
val_loss: 0.5753
val_loss: 0.5977

enter image description here

After this training has stopped. As far as I see there are no 2 consecutively increasing loss values at the end. Could someone give an explanation to this parameter-phenomena?

Upvotes: 10

Views: 12204

Answers (2)

INDRANUJ GANGAN
INDRANUJ GANGAN

Reputation: 33

Epoch 1      val_loss: 0.6811 <- current best
Epoch 2      val_loss: 0.6941 <-patience 1
Epoch 3      val_loss: 0.6532 <- current best # current best gets updated
Epoch 4      val_loss: 0.6546 <- patience 1
Epoch 5      val_loss: 0.6534 <-patience 2

Training will stop at epoch 5

Try this example in google colab for more intuitive understanding:https://colab.research.google.com/github/minsuk-heo/tf2/blob/master/jupyter_notebooks/06.DropOut_EarlyStopping.ipynb

Upvotes: 2

Thomas Jungblut
Thomas Jungblut

Reputation: 20969

There are three consecutively worse runs by loss, let's look at the numbers:

val_loss: 0.5921 < current best
val_loss: 0.5731 < current best
val_loss: 0.5956 < patience 1
val_loss: 0.5753 < patience 2
val_loss: 0.5977 < patience >2, stopping the training

You already discovered the min delta parameter, but I think it is too small to trigger here (you're off by 10x).

Upvotes: 21

Related Questions