The val_accuracy is higher than training accuracy, and the test accuracy is very low compared to both val_accuracy and train_accuracy

Question

I am training a CNN model where,

1. Training data=687 , validation data=102 , testing data=79
2. The validation accuracy is higher than training accuracy
3. The test accuracy is very low compared to both validation accuracy and training accuracy.
4. Validation loss is lower than training loss.

code snippet:

train_datagen = ImageDataGenerator(
                    rescale=1./255,
                    # rotation_range=30,
                    zoom_range=0.1,
                    horizontal_flip=True,
                    # vertical_flip=True,
                    # fill_mode='nearest',
                    validation_split=.15) # set validation split

val_datagen = ImageDataGenerator(rescale=1./255, validation_split=0.15)

train_generator = train_datagen.flow_from_directory(
    train_dir,
    target_size=(height, width),
    batch_size=batch_size,
    class_mode='categorical',
    seed=13
    )

validation_generator = val_datagen.flow_from_directory(
    train_dir, # same directory as training data
    target_size=(height, width),
    batch_size=batch_size,
    class_mode='categorical',
    seed=13,
    subset='validation'
    )
model = Sequential()
model.add(Conv2D(16,3,padding="same", activation="relu", input_shape=(height, width, 3)))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))

model.add(Conv2D(32,3,padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))

model.add(Conv2D(32, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))

# model.add(Conv2D(32, 3, padding="same", activation="relu", kernel_regularizer=l2(0.0001)))
# model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.4))

model.add(Conv2D(64, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same",))
# model.add(Dropout(0.2))

model.add(Conv2D(64, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))

model.add(Conv2D(128, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))

model.add(Conv2D(128, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))

model.add(Conv2D(256, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))


model.add(Flatten())
model.add(Dense(256,activation="relu"))
model.add(Dropout(.5))
# model.add(Dense(256,activation="relu"))
# model.add(Dropout(.5))
model.add(Dense(4, activation="softmax"))

model.summary()

Adam(learning_rate=0.0001, name='Adam')
model.compile(optimizer = 'Adam',loss = 'categorical_crossentropy',metrics = ['accuracy'])

I've done few things to solve this problem:

• I didn't used dropout layers between the conv layers
• I decreased the range of the data augmentation.
• I trained for longer period(the testing accuracy drops to 62% and the val_acc eventually reaches 100%).

What could be the cause of this issue and how can it be resolved? How can I display test images in Python that have high levels of inaccuracy?

Answer 1

You need to add subset='training' in train_generator. Right now, you are training on both training and validation data.

train_generator = train_datagen.flow_from_directory(
    train_dir,
    target_size=(height, width),
    batch_size=batch_size,
    class_mode='categorical',
    seed=13,
    subset='training'
    )