CODEWITHSUNDEEP
pythontensorflowconv-neural-network
Wassim Bouatay
Wassim Bouatay

Reputation: 193

Fluctuating Learning Curve

I am training a CNN model, here is the code.

model = tf.keras.models.Sequential([
    tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(300, 300,3)),
    tf.keras.layers.MaxPooling2D(2, 2),
    tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2, 2),
    tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2, 2),
    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2,2),
    tf.keras.layers.Flatten(input_shape=(300, 300)),
    tf.keras.layers.Dense(512, activation=tf.nn.relu),
    tf.keras.layers.Dense(256, activation=tf.nn.relu),
    tf.keras.layers.Dense(64, activation=tf.nn.relu),
    tf.keras.layers.Dense(1, activation='linear')
])

I am using Adam with 0.00003 learning rate, and training on 100 epochs.

However, The 10 epochs the validation starts to fluctuate between 0.16 and 0.22. ( I can't use early stopping because each time i retry the training the minimum is reached after a random number of epochs ).

Is this learning Curve normal ? what can i do to improve it ?

curve

Upvotes: 1

Views: 935

Answers (1)

Imtinan Azhar
Imtinan Azhar

Reputation: 1753

Yes, it is normal, no need to get worried, but for the future, I would suggest applying smoothing to the graphs, this will help you understand a lot better what is going on. For example, look at this graph I captured from tensorboard

enter image description here

The transparent graph is with the initial values, and the darker graph is a smoothened graph. the smoothened graph shows that the model is minimizing the loss, very haphazardly and slowly, but it is minimizing nonetheless.

Sometimes viewing a smoothened out graph can help better identify the training of the models.

Upvotes: 3

Related Questions