CODEWITHSUNDEEP
pythonkerastf.kerastransfer-learninglearning-rate
owise
owise

Reputation: 1075

Validation loss bounces randomly when training a Keras model regardless of the used optimiser

I am retraining InceptionV3 model on 200 images and I am using Adam optimiser: 

opt = Adam(lr=0.0001, decay=0.0001 / 100)

I noticed the loss bounces specially the validation. I thought that is down to the learning rate as I saw in some answers like Transfer Learning - Val_loss strange behaviour, and also Why is it possible to have low loss, but also very low accuracy, in a convolutional neural network? they were not helpful.

so I used RMSprop, but I had the same behaviour. Here is how the performance looks like:

enter image description here

Any suggestions why am I experiencing this and how to tackle it?

Upvotes: 0

Views: 206

Answers (1)

Susmit Agrawal
Susmit Agrawal

Reputation: 3764

Looking at your graphs, I don't think the network is actually learning anything.

I suggest you look into the following:

  1. Is there any 0'ed input in the images.

  2. Are the gradients either too large or too small.

  3. Are the gradients almost constant across multiple batches.

  4. Are the scales of all images the same.

  5. Are the classes properly encoded as one-hot vectors.

Upvotes: 1

Related Questions