Rendering a confusion matrix

Question

I am working with jupyterlab, specifically rendering a confusion matrix. However, when rendering the matrix, it seems as if there is something wrong because the figure is not fully rendered.

I already had installed the sklearn packages, but still the same problem. I tried different alternatives, but still rendering a snipped confusion matrix.

Below an example of a code that I know would render a proper confusion matrix.

from sklearn.metrics import classification_report, confusion_matrix
import itertools
import matplotlib.pyplot as plt

def plot_confusion_matrix(cm, classes,
                          normalize=False,
                          title='Confusion matrix',
                          cmap=plt.cm.Blues):
    """
    This function prints and plots the confusion matrix.
    Normalization can be applied by setting `normalize=True`.
    """
    if normalize:
        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
        print("Normalized confusion matrix")
    else:
        print('Confusion matrix, without normalization')

    print(cm)

    plt.imshow(cm, interpolation='nearest', cmap=cmap)
    plt.title(title)
    plt.colorbar()
    tick_marks = np.arange(len(classes))
    plt.xticks(tick_marks, classes, rotation=45)
    plt.yticks(tick_marks, classes)

    fmt = '.2f' if normalize else 'd'
    thresh = cm.max() / 2.
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        plt.text(j, i, format(cm[i, j], fmt),
                 horizontalalignment="center",
                 color="white" if cm[i, j] > thresh else "black")

    plt.tight_layout()
    plt.ylabel('True label')
    plt.xlabel('Predicted label')

# Compute confusion matrix
cnf_matrix = confusion_matrix(y_test, yhat, labels=[2,4])
np.set_printoptions(precision=2)

print (classification_report(y_test, yhat))

# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cnf_matrix, classes=['Benign(2)','Malignant(4)'],normalize= False,  title='Confusion matrix')

From the above code, I am obtaining this confusion matrix:

However, I expected to have a non snipped confusion matrix, such as:

credits: @Calvin Duy Canh Tran

UPDATE 2019-08-05:

To don't have doubts about the code used above, I used and additional reference: Instead, I tried the code that is one of the examples for the documentation for Confusion Matrix is scikit-learn. The link is this one https://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html

Prior to run the above-described code, I installed the correspondent module:

pip install -q scikit-plot

Unfortunately, the output continues rendering snipped matrixes (see the picture):

The correct output should be this one (ignore the orientation):

Answer 1

There seems to be a conflict between matplotlib version 3.1.1 and scikit-plot. Refer to this GitHub issue, which shows a similar problem.

Downgrading matplotlib to version 3.1.0 could be a immediate fix.

Answer 2

Do not pass the confusion matrix as input argument to the plotting function. You need to pass the y_test, y_pred and the confusion matrix will be calculated internally.

To plot it use this:

def plot_confusion_matrix(y_true, y_pred, classes,
                          normalize=False,
                          title=None,
                          cmap=plt.cm.Blues):
    """
    This function prints and plots the confusion matrix.
    Normalization can be applied by setting `normalize=True`.
    """
    if not title:
        if normalize:
            title = 'Normalized confusion matrix'
        else:
            title = 'Confusion matrix, without normalization'

    # Compute confusion matrix
    cm = confusion_matrix(y_true, y_pred)
    # Only use the labels that appear in the data
    classes = classes[unique_labels(y_true, y_pred)]
    if normalize:
        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
        print("Normalized confusion matrix")
    else:
        print('Confusion matrix, without normalization')

    print(cm)

    fig, ax = plt.subplots()
    im = ax.imshow(cm, interpolation='nearest', cmap=cmap)
    ax.figure.colorbar(im, ax=ax)
    # We want to show all ticks...
    ax.set(xticks=np.arange(cm.shape[1]),
           yticks=np.arange(cm.shape[0]),
           # ... and label them with the respective list entries
           xticklabels=classes, yticklabels=classes,
           title=title,
           ylabel='True label',
           xlabel='Predicted label')

    # Rotate the tick labels and set their alignment.
    plt.setp(ax.get_xticklabels(), rotation=45, ha="right",
             rotation_mode="anchor")

    # Loop over data dimensions and create text annotations.
    fmt = '.2f' if normalize else 'd'
    thresh = cm.max() / 2.
    for i in range(cm.shape[0]):
        for j in range(cm.shape[1]):
            ax.text(j, i, format(cm[i, j], fmt),
                    ha="center", va="center",
                    color="white" if cm[i, j] > thresh else "black")
    fig.tight_layout()
    return ax


# Plot non-normalized confusion matrix
plot_confusion_matrix(y_test, y_pred, classes=['Benign(2)','Malignant(4)'],normalize= False,  title='Confusion matrix')

instead of

plot_confusion_matrix(cnf_matrix, classes=['Benign(2)','Malignant(4)'],normalize= False,  title='Confusion matrix')

Reference: https://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html