CODEWITHSUNDEEP
pythontensorflowkerasloss-function
WVJoe
WVJoe

Reputation: 525

What does it mean for the loss in Keras to be different by orders of magnitude between models?

For context, I trained two separate autoencoders in Keras: one with a standard MSE loss function and one with a customized MSE loss function. When evaluating both models at a given point in training, they have very similar performance but the losses are very different.

My metric for performance is mean percent error. Both models are recreating the original image with mean error on the order of 3%. However, when these models were saved, the standard Keras MSE model had a loss less than 1.0 while the model with the customized MSE cost function had a loss on the order of 30.

If they perform on such a consistent level, why are the losses so drastically different?

Upvotes: 0

Views: 333

Answers (1)

Augusto Maillo
Augusto Maillo

Reputation: 180

Loss is just a scalar that says to model train which direction take to adjust weights. If you multiply a loss by a scalar, the result will be almost the same. Im not saying that the absolute value doesnt matter, it matters. But it's not the central point. Probably the difference in your case happens because Keras MSE do some normalizations that you dont.

Upvotes: 1

Related Questions