Train Accuracy increases, Train loss is stable, Validation loss Increases, Validation Accuracy is low and increases

Question

My neural network trainign in pytorch is getting very wierd.

I am training a known dataset that came splitted into train and validation. I'm shuffeling the data during training and do data augmentation on the fly.

I have those results:

Train accuracy start at 80% and increases

Train loss decreases and stays stable

Validation accuracy start at 30% but increases slowly

Validation loss increases

I have the following graphs to show:

1. How can you explain that the validation loss increases and the validation accuracy increases?

2. How can be such a big difference of accuracy between validation and training sets? 90% and 40%?

Update:

I balanced the data set. It is binary classification. It now has now 1700 examples from class 1, 1200 examples from class 2. Total 600 for validation and 2300 for training. I still see similar behavior:

**Can it be becuase I froze the weights in part of the network?

**Can it be becuase the hyperparametrs like lr?

Answer 1

Let me answer your 2nd question first. High accuracy on training data and low accuracy on val/test data indicates the model might not generalize well to infer real cases. That is what the validation process is all about. You need to finetune or even rebuild your model.
With regard to the first question, val loss might not necessarily correspond to the val accuracy. The model makes the prediction based on its model, and loss function calculates the difference between probablities of matrix and the target if you are using CrossEntropy function.

Answer 2

I found the solution: I had different data augmentation for training set and validation set. Matching them also increased the validation accuracy!

Answer 3

If the training set is very large in comparison to the validation set, you are more likely to overfit and learn the training data, which would make generalizing the model very difficult. I see your training accuracy is at 0.98 and your validation accuracy increases at a very slow rate, which would imply that you have overfit your training data.

Try reducing the number of samples in your training set to improve how well your model generalizes to unseen data.