CODEWITHSUNDEEP
machine-learningscikit-learnconfusion-matrix
Chinmay Nayak
Chinmay Nayak

Reputation: 273

Confusion matrix for multi-class case, estimation of all evaluation metrics

I am doing some self learning in ML.I am trying to train a K-NN model and my model gave me below 7X7 confusion matrix.I have manually calculated the Accuracy for two class(A and B) but not sure whether my approcah is correct or not.I would like to someone else there if confirm my result so that I will write a progamr to calcluate other 5 classes.

Confusion Matrx : 

  A      B    C    D   E    F     G =  7 classes
[[ 238    1    2    0   41   11    0]
 [   0   25    0    0    3    1    0]
 [  21    1   32    0   17    4    0]
 [   0    0    0    7    9    3    0]
 [   7    0    0    0 3633    8    0]
 [  44    0    4    1  397  256    1]
 [   4    0    0    0    7    2    3]]


Class-A 
tp = 238
fp = 76  (Rest of the 'a' column e.g 21+0+7+44+4 = 76)
tn = 4414


    [25    0    0    3    1    0]
    [1    32    0   17    4    0]
    [0     0    7    9    3    0]
    [0     0    0 3633    8    0]
    [0     4    1  397  256    1]
    [0     0    0    7    2    3]

  Sum of all elements will be true negative = 4414

  fn = 55 (Rest of the 'a' rows element e.g 1+2+0+41+11+0 =55 )

  So class-A  confusion matrix will something look like below

  tp fp | 238 76
        |
  fn tn | 55 4414

  Class A Accuracy = tp+tn/tp+tn+fp+fn = 4652/4768 = 0.97

  Class B 
  tp = 25
  fp = 2
  tn = 4752

  [  238       2    0   41   11    0]
  [  21       32    0   17    4    0]
  [   0        0    7    9    3    0]
  [   7        0    0 3633    8    0]
  [  44        4    1  397  256    1]
  [   4        0    0    7    2    3]

  fn = 4

  So class-B  confusion matrix will something look like below

  tp fp | 2 76
        |
  fn tn | 4 4752

 Class B Accuracy = tp+tn/tp+tn+fp+fn = 4754/4834 = 0.98

I have tried to search online but didn't find any calculation more than 2X2 matrix.

Upvotes: 1

Views: 356

Answers (1)

seralouk
seralouk

Reputation: 33197

For a multi-class case this can be used:

import numpy as np

cnf_matrix = np.array([[13,  0,  0],
                       [ 0, 10,  6],
                       [ 0,  0,  9]])

FP = cnf_matrix.sum(axis=0) - np.diag(cnf_matrix)  
FN = cnf_matrix.sum(axis=1) - np.diag(cnf_matrix)
TP = np.diag(cnf_matrix)
TN = cnf_matrix.sum() - (FP + FN + TP)

FP = FP.astype(float)
FN = FN.astype(float)
TP = TP.astype(float)
TN = TN.astype(float)


# Sensitivity, hit rate, recall, or true positive rate
TPR = TP/(TP+FN)
# Specificity or true negative rate
TNR = TN/(TN+FP) 
# Precision or positive predictive value
PPV = TP/(TP+FP)
# Negative predictive value
NPV = TN/(TN+FN)
# Fall out or false positive rate
FPR = FP/(FP+TN)
# False negative rate
FNR = FN/(TP+FN)
# False discovery rate
FDR = FP/(TP+FP)

# Overall accuracy
ACC = (TP+TN)/(TP+FP+FN+TN)

The idea behind the code: These metrics are represented graphically for a general case with a lot of classes in the following image.

Multiclass Confusion Matrix

Upvotes: 2

Related Questions