CODEWITHSUNDEEP
machine-learningscikit-learndeep-learningdata-science
blue-sky
blue-sky

Reputation: 53896

Is this an example of over-fitting?

Training a deep learning model after approx 100 epochs : 

Train accuracy : 93 %
Test accuracy : 54 %

then training accuracy increases and test accuracy decreases : 

Train accuracy : 94 %
Test accuracy : 53 %
Train accuracy : 95 %
Test accuracy : 52 %
Train accuracy : 96 %
Test accuracy : 51 %

For initial version of the model we are satisfied with 54% accuracy but I don't know what the meaning of training accuracy increasing, test accuracy decreasing other than the model is overfitting. Should I stop training the model and use trained parameters when max test accuracy is achieved, in this case 54% ?

What knowledge can I gain from the observation of training accuracy increasing & test accuracy decreasing?, is this an example of stronger over-fitting ?

Upvotes: 0

Views: 148

Answers (2)

Dan9ie
Dan9ie

Reputation: 1

Yes definitely overfitting, when I first started building logistic regressions in SAS, we used to have a thumb rule of having a model with train and test performances not more than 10% off each other.

Another way could be to use k-fold and get balanced performance across all folds.

Overall it implies that the model is stable, we are fitting it to the actual data trends and not fitting it to the noise.

Upvotes: 0

Andreas Storvik Strauman
Andreas Storvik Strauman

Reputation: 1655

Yes this is definitely overfitting. You should terminate the training procedure at the point where the test accuracy stops increasing. By the numbers you show, your model is actually overfitting a lot. You should consider adding regularization to possibly increase the test accuracy.

(me adding): regularization is, as @Djib2011, says the way to go to help prevent overfitting. You could look into e.g. L2 or Dropout which are amongst the most common ones.

The question was answered in the comments and, since no one wrote an answer, I made this answer a community wiki answer. This is to remove this question from the unanswered list. The original answer was by @Djib2011 . The OP is encouraged to select this as the answer to remove the questions status as unanswered. (If the person who answered in the comments decides to make an answer the OP can, and should, select that answer instead).

Upvotes: 2

Related Questions