CNN giving incorrect predictions using predict_classes

Question

I am still in the learning mode with keras and CNNs. I feel like I am understanding basic points, but my execution is difficult. I created a Sequential classifier used for the dog/cat image dataset. I used fit_generator to get the following (code for this under):

Epoch 1/5
8000/8000 [==============================] - 1942s 243ms/step - loss: 0.3658 - acc: 0.8299 - val_loss: 0.6998 - val_acc: 0.7785] - ETA: 24:40 - loss: 0.6010 - acc: 0.6705
Epoch 2/5
8000/8000 [==============================] - 1829s 229ms/step - loss: 0.1266 - acc: 0.9522 - val_loss: 0.9218 - val_acc: 0.7731
Epoch 3/5
8000/8000 [==============================] - 1806s 226ms/step - loss: 0.0689 - acc: 0.9759 - val_loss: 1.2006 - val_acc: 0.7813
Epoch 4/5
8000/8000 [==============================] - 1936s 242ms/step - loss: 0.0504 - acc: 0.9830 - val_loss: 1.2396 - val_acc: 0.7748- ETA: 18:07 - loss: 0.0548 - acc: 0.9817
Epoch 5/5
8000/8000 [==============================] - 2259s 282ms/step - loss: 0.0393 - acc: 0.9870 - val_loss: 1.3916 - val_acc: 0.7818

Code used to produce results above:

# Importing the Keras libraries and packages
import os
from keras.models import Sequential, load_model
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense

from keras.preprocessing import image
from keras.preprocessing.image import ImageDataGenerator

#create classifying sequential neural network
classifier = Sequential()

classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Flatten())
classifier.add(Dense(units = 128, activation = 'relu')) #ReLU stands for Rectified Linear Unit. It takes a real-valued input and thresholds it at zero (replaces negative values with zero)

#initialise our output layer, which should contain only one node, as it is 
#binary classification. Single node gives us a binary output of either a Cat or Dog.
classifier.add(Dense(units = 1, activation = 'sigmoid')) 

#compile CNN model
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])

#CREATE IMAGE DATA GENERATORS---
#perform image augmentations, essentially synthesising training data 
train_datagen = ImageDataGenerator(rescale = 1./255, 
                                   shear_range = 0.2, 
                                   zoom_range = 0.2, 
                                   horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale = 1./255)

training_set = train_datagen.flow_from_directory('training_set', 
                                                 target_size = (64, 64), 
                                                 batch_size = 32, 
                                                 class_mode = 'binary')

#create test set for training, by feeding generated data from images using the test directory
test_set = test_datagen.flow_from_directory('test_set', 
                                            target_size = (64, 64), 
                                            batch_size = 32, 
                                            class_mode = 'binary')


#fit the training set to model        
classifier.fit_generator(training_set, 
                         steps_per_epoch = 8000, 
                         epochs = 5, 
                         validation_data = test_set, 
                         validation_steps = 2000)

#save the model for further use
classifier.save('classifier_v1.h5')

from here, I then used the load_model function to recall my classifier and tried to predict whether dog or cat on a test set of 5 pictures. The classifier only results to 1, not matter what I do.

from keras.models import load_model
from keras.preprocessing import image
import numpy as np

classifier = load_model('classifier_v1.h5')
data_path = r'C:\Users\aneja\Documents\Python Scripts\CNN\Cats-Dogs\test_2'
image_list = [x for x in os.listdir(data_path) if '.jpg' in x]

#loop to test through test images
for jpeg in image_list:
    #load a test image
    img = image.load_img(os.path.join(data_path, jpeg), target_size=(64,64))

    #process image to extract numpy arrays
    y = image.img_to_array(img)
    x = np.expand_dims(y, axis=0)

    #predict!   1 = dog   0 = cat
    images = np.vstack([x])
    classes = classifier.predict_classes(images, batch_size=10)
    if classes[0][0]==1:
        print('The {} file is predicted to be a dog'.format(jpeg))
    elif classes[0][0]==0:
        print('The {} file is prediected to be a cat'.format(jpeg))
    else:
        print('Yea, I did something wrong')

I also get the same results when predict is used instead of predict_classes. I hope I am not too far away from the solution, but I fear that I have fundamentally misunderstood concepts. Can anyone offer any help as to why my classifier seems to ALWAYS classify the class as 1?

Answer 1

Here are a few suggestions:

First and foremost, check to make sure that your "steps_per_epoch = 8000" and "validation_steps=2000" actually are the correct values. From my experience, incorrect numbers greatly harm the model's performance. Most people ensure these values are correct by doing something like: generator_train.samples/generator_train.batch_size

Second, increase the number of Conv2D layers. Typically, one layer is insufficient for a strong model. If you find that your computer cannot quickly train your model, then use Google Colaboratory, a free GPU and cloud based site that can allow you to do this.

Third, increase your test size. 5 images is not enough for you to confidently assess your model's performance.

Fourth, going off what S.Mohsen sh said, if you find that "predict" gives just returns "1s", then it means that your model has not been properly trained, so try my first two suggestions.

Hope this has helped!