CODEWITHSUNDEEP
machine-learningkerastensorflow2.0
than_g
than_g

Reputation: 117

Keras loss changes on epoch end

I am training a model in Keras (tensorflow2 backend) using the imagedatagenerator class to train on batches. I have noticed that when the second epoch starts the loss value is really smaller than the value at the end of the first epoch.

Here is what I mean:

Keras loss

Note that the starting value in the second epoch is around the value that you are seeing in the screenshot.

Does anyone knows why that happens?

Does keras updates again the weights when all batches are processed ?

Upvotes: 2

Views: 401

Answers (1)

Nikola Dim
Nikola Dim

Reputation: 846

The loss is expected to be smaller, but your surprise to the extent of it is understandable.

The reason the second epoch has such a lower loss is because during the first epoch your model makes mistakes and yields great losses - which get better and better. Keras displays the mean loss over all instances in an epoch.

So if the model made mistakes on the first 90% of the training set in epoch, and then was perfect for the last 10% of the data, the loss would still be very large because it's mean loss.

Then, at the start of 2nd epoch, model is already better at predicting, so the mean loss is lower.

Upvotes: 3

Related Questions