CODEWITHSUNDEEP
machine-learningneural-network
Bastian
Bastian

Reputation: 1593

MSE during Neural Network training

Is it possible that the MSE increases during training?

I'm currently calculating the MSE of the validation set per epoch and at a certrain point, the MSE starts to increase instead of decreasing. Does someone has an explanation for this behavior?

Upvotes: 2

Views: 1737

Answers (1)

Will Glück
Will Glück

Reputation: 1282

Answering your question: Yes, it is possible.

If you are using regularization or estochastic training it is normal some ups and downs on the MSE while training.

Some possible reasons to the problem

  1. You are using a learning rate too high, which let to the problem of overshooting the local minima of the cost function.

  2. The neural network is overfitting. Traning too much and loosing its capabilities to generalize.

What you can try:

  1. When this starts to happen, reduce your learning rate.

  2. Apply some kind of regularization on your network, like dropout, to avoid overfitting.

Upvotes: 5

Related Questions