CODEWITHSUNDEEP
neural-networkkerasregressionmse
user153245
user153245

Reputation: 287

Training keras model; why mae decreases while mse increases?

I am training a model using keras on a regression problem. When I investigate the loss and metrics during training, sometimes mean absolute error (mae) decreases at the end of an epoch, while mean square error (mse) increases. I set mae as loss and mse as metric.

Is it OK? Or is there any problem with the setting? Thanks

Upvotes: 1

Views: 4221

Answers (2)

Paweł Kozielski-Romaneczko
Paweł Kozielski-Romaneczko

Reputation: 1118

Using mae as loss leads to predicting median value, while using mse leads to predicting average value. You can check this post: https://stats.stackexchange.com/questions/355538/why-does-minimizing-the-mae-lead-to-forecasting-the-median-and-not-the-mean for more detailed explanation.

Upvotes: 0

KiraMichiru
KiraMichiru

Reputation: 1018

MSE and MAE are different metrics. A decrease in the one does not imply a decrease in the other. Consider the following toy example for the size-2 output values of a network with the target value as Target: [0,0]

  • Timestep 1: Output: [2,2], MAE: 2, MSE: 4
  • Timestep 2: Output: [0,3], MAE: 1.5, MSE: 4.5

So MAE decreased while MSE increased. Given that you are optimizing for MAE and only monitor MSE, your observation is perfectly fine and does not imply any problem.

Upvotes: 4

Related Questions