CODEWITHSUNDEEP
pythonkerasimage-segmentation
SagRU
SagRU

Reputation: 448

How to create a confusion matrices for an image segmentation task?

I'm dealing with binary image segmentation problem. I've successfully compiled and trained the model. Now I'm trying to achieve two goals:

  1. Get a total confusion matrix for a test set (reason: understand proportions of false positives and false negatives)
  2. Get an individual confusion matrix for every image in a test set (reason: find and analyze images that drag model performance down)

As far as I understand, confusion_matrix from scikit-learn package can help with a total confusion matrix, but I can't make it work with my custom data generator. According to docs, here's a code for confusion_matrix:

sklearn.metrics.confusion_matrix(y_true, y_pred, *, labels=None, sample_weight=None, normalize=None)

I don't understand how to retrieve y_true with my custom data generator:

def learn_generator(templates_folder, masks_folder, image_width, batch_size, shuffle=True):
    """Generate individual batches form dataset"""
    counter = 0
    images_list = os.listdir(templates_folder)
    if shuffle:
        random.shuffle(images_list)
    while True:
        templates_pack = np.zeros((batch_size, image_width, image_width, 3)).astype('float')
        masks_pack = np.zeros((batch_size, image_width, image_width, 1)).astype('float')
        for i in range(counter, counter + batch_size):
            template = cv2.imread(templates_folder + '/' + images_list[i]) / 255.
            templates_pack[i - counter] = template
            mask = cv2.imread(masks_folder + '/' + images_list[i], cv2.IMREAD_GRAYSCALE) / 255.
            mask = np.expand_dims(mask, axis=2)
            masks_pack[i - counter] = mask
        counter += batch_size
        if counter + batch_size >= len(images_list):
            counter = 0
            if shuffle:
                random.shuffle(images_list)
        yield templates_pack, masks_pack

test_templates_path = "E:/Project/images/all_templates/test"
test_masks_path = "E:/Project/images/all_masks/test"
TEST_SET_SIZE = len(os.listdir(test_templates_path))
IMAGE_WIDTH = 512
BATCH_SIZE = 4
TEST_STEPS = TEST_SET_SIZE / BATCH_SIZE

test_generator = learn_generator(test_templates_path, test_masks_path, IMAGE_WIDTH, batch_size=BATCH_SIZE)
Y_pred = model.predict_generator(test_generator, steps=TEST_STEPS)
y_pred = np.argmax(Y_pred, axis=1)

y_true = ???

As for individual confusion matrices, no ideas at all... Any help is appreciated.

Upvotes: 3

Views: 8456

Answers (2)

Lue Mar
Lue Mar

Reputation: 472

I suppose it's too late for you, but maybe this could help somebody else :

I did achive this by using the definition of a confusion matrix, by calculation True Positives, True Negatives, False Positives, False negatives.

This code works only for binary segmentation, assuming that "1" is the output for "positive" and "0" for "negative"...

    import seaborn as sns

    FP = len(np.where(Y_pred - Y_val  == 1)[0])
    FN = len(np.where(Y_pred - Y_val  == -1)[0])
    TP = len(np.where(Y_pred + Y_val ==2)[0])
    TN = len(np.where(Y_pred + Y_val == 0)[0])
    cmat = [[TP, FN], [FP, TN]]

    plt.figure(figsize = (6,6))
    sns.heatmap(cmat/np.sum(cmat), cmap="Reds", annot=True, fmt = '.2%', square=1,   linewidth=2.)
    plt.xlabel("predictions")
    plt.ylabel("real values")
    plt.show()

confusion_matrix

Upvotes: 6

aunsid
aunsid

Reputation: 397

https://www.kite.com/blog/python/image-segmentation-tutorial/

I think you could change the code from this website for two classes and apply it for your use case.

Additionally, I think this answer does provide the implementation of sklearn confusion matrix method Faster method of computing confusion matrix?

Upvotes: 0

Related Questions