CNN- Trying to run a confusion matrix using seaborn.heatmap

Question

I've been trying to run a confusion matrix after my CNN model ran.

My model is classifying dogs/bunnies.

The following is what I did:

I placed the photos of each class (dogs/bunnies) in separate folders inside two folders: training and testing.

Training directory-> Bunny directory -> bunny images

Training directory-> Puppy directory -> puppy images

Testing directory-> Bunny directory -> bunny images

Testing directory-> Puppy directory -> puppy images

I used the following code to get the images from the folders:

training_data = train_datagen.flow_from_directory('./images/train',
                                             target_size = (28, 28),
                                             batch_size = 86,
                                             class_mode = 'binary',
                                             color_mode='rgb',
                                             classes=None)


test_data = test_datagen.flow_from_directory('./images/test',
                                        target_size = (28, 28),
                                        batch_size = 86,
                                        class_mode = 'binary',
                                        color_mode='rgb',
                                        classes=None)

I used the following code to seperate the images into training/val.

data_generator = ImageDataGenerator(
    validation_split=0.2,
    rescale=1./255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True,
)

train_generator = data_generator.flow_from_directory(
    './images/train',
    target_size = (28, 28),
    batch_size = 86,
    class_mode = 'binary',
    color_mode='rgb',
    classes=None, subset="training"
)

validation_generator = data_generator.flow_from_directory(
    './images/train',
    target_size = (28, 28),
    batch_size = 86,
    class_mode = 'binary',
    color_mode='rgb',
    classes=None, subset="validation"
)

history=classifier.fit_generator(
    train_generator,
    steps_per_epoch = (8000 / 86),
    epochs = 2,
    validation_data = validation_generator,
    validation_steps = 8000/86,
    callbacks=[learning_rate_reduction]
)

When I tried to run confusion_matrix(validation_data) I get this error:

TypeError: confusion_matrix() missing 1 required positional argument: 'y_pred'

And when I run

#Confusion matrix
def plot_confusion_matrix(cm, classes,
                          normalize=False,
                          title='Confusion matrix',
                          cmap=plt.cm.Blues):
    """
    This function prints and plots the confusion matrix.
    Normalization can be applied by setting `normalize=True`.
    """
    plt.imshow(cm, interpolation='nearest', cmap=cmap)
    plt.title(title)
    plt.colorbar()
    tick_marks = np.arange(len(classes))
    plt.xticks(tick_marks, classes, rotation=45)
    plt.yticks(tick_marks, classes)

    if normalize:
        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]

    thresh = cm.max() / 2.
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        plt.text(j, i, cm[i, j],
                 horizontalalignment="center",
                 color="white" if cm[i, j] > thresh else "black")

    plt.tight_layout()
    plt.ylabel('True label')
    plt.xlabel('Predicted label')

# Predict the values from the validation dataset
Y_pred = classifier.predict(training_data)
# Convert predictions classes to one hot vectors 
Y_pred_classes = np.argmax(Y_pred,axis = 1) 
# Convert validation observations to one hot vectors
Y_true = np.argmax(training_data,axis = 1) 
# compute the confusion matrix
confusion_mtx = confusion_matrix(Y_true, Y_pred_classes) 
# plot the confusion matrix
plot_confusion_matrix(confusion_mtx, classes = range(10))

sns.heatmap(confusion_mtx, annot=True, fmt='d')

I get the following error

AttributeError: 'DirectoryIterator' object has no attribute 'ndim'

Answer 1

As I understood you want to validate your classifier model using confusion matrix and heatmap. I have also made validation on Spam text classification so this is what you can do,

For confusion matrix,

from sklearn.metrics import confusion_matrix
conf_mat = confusion_matrix(y_test, y_pred)
print(conf_mat)

For Heatmap,

import seaborn as sns
conf_mat = confusion_matrix(y_test, y_pred)
conf_mat_normalized = conf_mat.astype('float') / conf_mat.sum(axis=1)[:, np.newaxis]
sns.heatmap(conf_mat_normalized)
plt.ylabel('True label')
plt.xlabel('Predicted label')

Simply means confusion matrix requires two parameters (Your actual truth label and predicted label lists)

Hope it will help you.