CODEWITHSUNDEEP
pythondeep-learningconv-neural-networktf.keras
Saankhya Mondal
Saankhya Mondal

Reputation: 151

Validation accuracy not improving after reaching 90%

I have around 14,000 images for face detection binary classification task. Around 12,000 images are used for training and 2,200 for validation. My CNN architecture is as follows -

regularizer = tf.keras.regularizers.l2(l=0.001)
init = tf.initializers.he_uniform()
model = tf.keras.models.Sequential([

    tf.keras.layers.Conv2D(filters = 32, kernel_size = (3, 3), strides = (1, 1), padding='same', activation='relu', kernel_initializer=init, kernel_regularizer=regularizer, input_shape=(ht, wd, 3)),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=(2, 2)),

    tf.keras.layers.Conv2D(64, (5, 5), (1, 1), padding='same', activation='relu', kernel_initializer=init, kernel_regularizer=regularizer),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.MaxPooling2D((2, 2), (2, 2)),

    tf.keras.layers.Conv2D(64, (3, 3), (1, 1), padding='same', activation='relu', kernel_initializer=init, kernel_regularizer=regularizer),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.MaxPooling2D((2, 2), (2, 2)),

    tf.keras.layers.Conv2D(128, (3, 3), (1, 1), padding='same', activation='relu', kernel_initializer=init, kernel_regularizer=regularizer),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.MaxPooling2D((2, 2), (2, 2)),

    tf.keras.layers.Conv2D(128, (3, 3), (1, 1), padding='same', activation='relu', kernel_initializer=init, kernel_regularizer=regularizer),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.MaxPooling2D((2, 2), (2, 2)),

    tf.keras.layers.Flatten(),

    tf.keras.layers.Dense(256, activation = 'relu', kernel_regularizer= regularizer),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.Dropout(0.25),
    tf.keras.layers.Dense(1, activation = 'sigmoid')
])

My input image dimensions are 150*180*3. The CNN has about 900k - 1M parameters. I'm using batch size of 16/32. Also, my learning rate(initial lr=0.001) scheduler of Adam optimizer is like this

reduce_lr_2 = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', 
                                             factor=0.2, 
                                             patience=2, 
                                             verbose=0, 
                                             mode='auto',
                                             min_delta=0.0001, 
                                             cooldown=1,
                                             min_lr=0)

I applied data augmentation of many types. Training accuracy I reach is about 95-96% and validation accuracy I achieve is about 90%. I've tried changing a lot of hyper parameters but the validation accuracy remains stuck at around 90%. Is there any way to improve the validation accuracy?

Upvotes: 1

Views: 2035

Answers (1)

Jeyanth Krishna - Intel
Jeyanth Krishna - Intel

Reputation: 116

There are 4 ways to improve deep learning performance:

  1. Improve Performance With Data.
  2. Improve Performance With Algorithms.
  3. Improve Performance With Algorithm Tuning.
  4. Improve Performance With Ensembles.

Improve Performance With Data:

Get More Data. Invent More Data. Rescale Your Data. Transform Your Data. Feature Selection

Improve Performance With Algorithms

Spot-Check Algorithms: Maybe your chosen algorithms is not the best for your problem. Resampling Methods: You must know how good your models are. Is your estimate of the performance of your models reliable?

Improve Performance With Algorithm Tuning

some ideas on tuning your neural network algorithms in order to get more out of them.

Diagnostics. Weight Initialization. Learning Rate. Activation Functions. Network Topology. Batches and Epochs. Regularization. Optimization and Loss. Early Stopping.

Improve Performance With Ensembles

three general areas of ensembles you may want to consider:

Combine Models. Combine Views. Stacking.

check below link for further information: https://machinelearningmastery.com/improve-deep-learning-performance/

Upvotes: 2

Related Questions