VGG16 Net Validation Accuracy

Question

I tried classifying 16 image class labelled data. I got an validation accuracy as 60% but when i try to predict the class label on the validation data. My accuracy goes to 6%.

load the weight from the saved file

```
model.load_weights("weights_copy.best.hdf5")
valid_generator.reset()
nb_validation_samples = len(valid_generator.classes)
pred= model.predict_generator(valid_generator, nb_validation_samples//batch_size)
predicted_class_indices=np.argmax(pred,axis=1)
labels=(valid_generator.class_indices)
labels2=dict((v,k) for k,v in labels.items())
predictions=[labels2[k] for k in predicted_class_indices]
true_labels=[labels2[k] for k in valid_generator.classes]
```

Could you please tell me why is this happening?

Answer 1

I couldn't find the code that you have implemented to manually calculate the Accuracy after getting the predictions using predict_generator.

Below is the simple Cat and Dog Classification program that I have ran for one epoch-

1. Validation accuracy is displayed after end of the epoch.
2. Then save the weights using save_weights.
3. Later loaded the model using load_weights.
4. Built the Confusion matrix using sklearn's confusion_matrix and computed accuracy.

The validation accuracy computed after end of the epoch and manually computed accuracy are matching.

Code -

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.optimizers import Adam

import os
import numpy as np
import matplotlib.pyplot as plt

_URL = 'https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip'

path_to_zip = tf.keras.utils.get_file('cats_and_dogs.zip', origin=_URL, extract=True)

PATH = os.path.join(os.path.dirname(path_to_zip), 'cats_and_dogs_filtered')

train_dir = os.path.join(PATH, 'train')
validation_dir = os.path.join(PATH, 'validation')

train_cats_dir = os.path.join(train_dir, 'cats')  # directory with our training cat pictures
train_dogs_dir = os.path.join(train_dir, 'dogs')  # directory with our training dog pictures
validation_cats_dir = os.path.join(validation_dir, 'cats')  # directory with our validation cat pictures
validation_dogs_dir = os.path.join(validation_dir, 'dogs')  # directory with our validation dog pictures

num_cats_tr = len(os.listdir(train_cats_dir))
num_dogs_tr = len(os.listdir(train_dogs_dir))

num_cats_val = len(os.listdir(validation_cats_dir))
num_dogs_val = len(os.listdir(validation_dogs_dir))

total_train = num_cats_tr + num_dogs_tr
total_val = num_cats_val + num_dogs_val

batch_size = 128
epochs = 1
IMG_HEIGHT = 150
IMG_WIDTH = 150

train_image_generator = ImageDataGenerator(rescale=1./255,brightness_range=[0.5,1.5]) # Generator for our training data
validation_image_generator = ImageDataGenerator(rescale=1./255,brightness_range=[0.5,1.5]) # Generator for our validation data

train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,
                                                           directory=train_dir,
                                                           shuffle=True,
                                                           target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                           class_mode='binary')

val_data_gen = validation_image_generator.flow_from_directory(batch_size=batch_size,
                                                              directory=validation_dir,
                                                              target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                              class_mode='binary')

model = Sequential([
    Conv2D(16, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
    MaxPooling2D(),
    Conv2D(32, 3, padding='same', activation='relu'),
    MaxPooling2D(),
    Conv2D(64, 3, padding='same', activation='relu'),
    MaxPooling2D(),
    Flatten(),
    Dense(512, activation='relu'),
    Dense(1)
])

optimizer = 'SGD'

model.compile(optimizer=optimizer, 
          loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
          metrics=['accuracy'])

history = model.fit_generator(
          train_data_gen,
          steps_per_epoch=total_train // batch_size,
          epochs=epochs,
          validation_data=val_data_gen,
          validation_steps=total_val // batch_size)

# Save the weights
model.save_weights('my_model.hdf5')

from sklearn.metrics import confusion_matrix

# Load the weights
model.load_weights('my_model.hdf5')

# Reset 
val_data_gen.reset()

# Predictions
pred = model.predict_generator(val_data_gen)
predicted = np.argmax(pred,axis=1)

# Actual Labels
labels = val_data_gen.classes

# Compute Accuracy
conf_mat = confusion_matrix(predicted, labels)
acc = np.sum(conf_mat.diagonal()) / np.sum(conf_mat)
print('Validation accuracy: {} %'.format(acc*100))

Output -

Found 2000 images belonging to 2 classes.
Found 1000 images belonging to 2 classes.
15/15 [==============================] - 29s 2s/step - loss: 0.6917 - accuracy: 0.4952 - val_loss: 0.6906 - val_accuracy: 0.5045
Validation accuracy: 50.0 %