s T
s T

Reputation: 47

How to find the ROC curve and AUC score of CNN model (keras)

I am new to deep learning. I am trying to generate ROC curve for the following code. I am using keras. The class size is 10 and the image are RGB image of size 100* 100* 3.

I went through [This link][1]. My problem is also same but I could not find the true labels. I am new in this field so please help me. I also looked on [This for true label][2].

The code snippet of my program is:

target_size=(100,100,3)

train_generator = train_datagen.flow_from_directory('path',
    target_size=target_size[:-1],
    batch_size=16,
    class_mode='categorical',
    subset='training',
    seed=random_seed)

valid_generator = ...

test_generator = ...
n_classes = len(set(train_generator.classes))



input_layer = keras.layers.Input(shape=target_size)

conv2d_1 = keras.layers.Conv2D(filters=64, kernel_size=(3,3), strides=1, padding='same', 
activation='relu',
                       kernel_initializer='he_normal')(input_layer)

batchnorm_1 = keras.layers.BatchNormalization()(conv2d_1)
maxpool1=keras.layers.MaxPool2D(pool_size=(2,2))(batchnorm_1)


conv2d_2 = keras.layers.Conv2D(filters=32, kernel_size=(3,3), strides=1, padding='same', 
activation='relu',
                       kernel_initializer='he_normal')(maxpool1)
batchnorm_2 = keras.layers.BatchNormalization()(conv2d_2)

maxpool2=keras.layers.MaxPool2D(pool_size=(2,2))(batchnorm_2)


flatten = keras.layers.Flatten()(maxpool2)
dense_1 = keras.layers.Dense(256, activation='relu')(flatten)

dense_2 = keras.layers.Dense(n_classes, activation='softmax')(dense_1)



model = keras.models.Model(input_layer, dense_3)

model.compile(optimizer=keras.optimizers.Adam(0.001),
      loss='categorical_crossentropy',
      metrics=['acc'])
model.summary()

model.fit_generator(generator=train_generator, validation_data=valid_generator,
            epochs=200)
            
score = model.evaluate_generator(test_generator)

print(score)

Now please help me in getting AUC score and ROC Curve. [1]: How to find the ROC curve and AUC score of this CNN model (keras) [2]: Getting true labels for keras predictions

Upvotes: 2

Views: 3698

Answers (2)

Virtuall.Kingg
Virtuall.Kingg

Reputation: 194

Add this code. Hope it works.

import numpy as np
from sklearn import metrics

x, y = test_generator.next()
prediction = model.predict(x)

predict_label1 = np.argmax(prediction, axis=-1)
true_label1 = np.argmax(y, axis=-1)

y = np.array(true_label1)

scores = np.array(predict_label1)
fpr, tpr, thresholds = metrics.roc_curve(y, scores, pos_label=9)
roc_auc = metrics.auc(fpr, tpr)


plt.figure()
lw = 2
plt.plot(fpr, tpr, color='darkorange',
 lw=lw, label='ROC curve (area = %0.2f)' % roc_auc)
plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver operating characteristic (ROC)')
plt.legend(loc="lower right")
plt.show()

Upvotes: 1

Swapnil Pote
Swapnil Pote

Reputation: 136

The score function does not provide roc and auc score by default we have to calculate separately. You can check following code snipet to calculate roc and auc score and plot there values.

from sklearn.metrics import roc_curve
y_pred_keras = model.predict(X_test).ravel()
fpr_keras, tpr_keras, thresholds_keras = roc_curve(y_test, y_pred_keras)

from sklearn.metrics import auc
auc_keras = auc(fpr_keras, tpr_keras)

import matplotlib.pyplot as plt
plt.figure(1)
plt.plot([0, 1], [0, 1], 'k--')
plt.plot(fpr_keras, tpr_keras, label='Keras (area = {:.3f})'.format(auc_keras))
plt.plot(fpr_rf, tpr_rf, label='RF (area = {:.3f})'.format(auc_rf))
plt.xlabel('False positive rate')
plt.ylabel('True positive rate')
plt.title('ROC curve')
plt.legend(loc='best')
plt.show()

Upvotes: 0

Related Questions