Reputation: 47
I am new to deep learning. I am trying to generate ROC curve for the following code. I am using keras. The class size is 10 and the image are RGB image of size 100* 100* 3.
I went through [This link][1]. My problem is also same but I could not find the true labels. I am new in this field so please help me. I also looked on [This for true label][2].
The code snippet of my program is:
target_size=(100,100,3)
train_generator = train_datagen.flow_from_directory('path',
target_size=target_size[:-1],
batch_size=16,
class_mode='categorical',
subset='training',
seed=random_seed)
valid_generator = ...
test_generator = ...
n_classes = len(set(train_generator.classes))
input_layer = keras.layers.Input(shape=target_size)
conv2d_1 = keras.layers.Conv2D(filters=64, kernel_size=(3,3), strides=1, padding='same',
activation='relu',
kernel_initializer='he_normal')(input_layer)
batchnorm_1 = keras.layers.BatchNormalization()(conv2d_1)
maxpool1=keras.layers.MaxPool2D(pool_size=(2,2))(batchnorm_1)
conv2d_2 = keras.layers.Conv2D(filters=32, kernel_size=(3,3), strides=1, padding='same',
activation='relu',
kernel_initializer='he_normal')(maxpool1)
batchnorm_2 = keras.layers.BatchNormalization()(conv2d_2)
maxpool2=keras.layers.MaxPool2D(pool_size=(2,2))(batchnorm_2)
flatten = keras.layers.Flatten()(maxpool2)
dense_1 = keras.layers.Dense(256, activation='relu')(flatten)
dense_2 = keras.layers.Dense(n_classes, activation='softmax')(dense_1)
model = keras.models.Model(input_layer, dense_3)
model.compile(optimizer=keras.optimizers.Adam(0.001),
loss='categorical_crossentropy',
metrics=['acc'])
model.summary()
model.fit_generator(generator=train_generator, validation_data=valid_generator,
epochs=200)
score = model.evaluate_generator(test_generator)
print(score)
Now please help me in getting AUC score and ROC Curve. [1]: How to find the ROC curve and AUC score of this CNN model (keras) [2]: Getting true labels for keras predictions
Upvotes: 2
Views: 3698
Reputation: 194
Add this code. Hope it works.
import numpy as np
from sklearn import metrics
x, y = test_generator.next()
prediction = model.predict(x)
predict_label1 = np.argmax(prediction, axis=-1)
true_label1 = np.argmax(y, axis=-1)
y = np.array(true_label1)
scores = np.array(predict_label1)
fpr, tpr, thresholds = metrics.roc_curve(y, scores, pos_label=9)
roc_auc = metrics.auc(fpr, tpr)
plt.figure()
lw = 2
plt.plot(fpr, tpr, color='darkorange',
lw=lw, label='ROC curve (area = %0.2f)' % roc_auc)
plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver operating characteristic (ROC)')
plt.legend(loc="lower right")
plt.show()
Upvotes: 1
Reputation: 136
The score function does not provide roc and auc score by default we have to calculate separately. You can check following code snipet to calculate roc and auc score and plot there values.
from sklearn.metrics import roc_curve
y_pred_keras = model.predict(X_test).ravel()
fpr_keras, tpr_keras, thresholds_keras = roc_curve(y_test, y_pred_keras)
from sklearn.metrics import auc
auc_keras = auc(fpr_keras, tpr_keras)
import matplotlib.pyplot as plt
plt.figure(1)
plt.plot([0, 1], [0, 1], 'k--')
plt.plot(fpr_keras, tpr_keras, label='Keras (area = {:.3f})'.format(auc_keras))
plt.plot(fpr_rf, tpr_rf, label='RF (area = {:.3f})'.format(auc_rf))
plt.xlabel('False positive rate')
plt.ylabel('True positive rate')
plt.title('ROC curve')
plt.legend(loc='best')
plt.show()
Upvotes: 0