Evaluating the performance of one class SVM

Question

I have been trying to evaluate the performance of my one-class SVM. I have tried plotting an ROC curve using scikit-learn, and the results have been a bit bizarre.

X_train, X_test = train_test_split(compressed_dataset,test_size = 0.5,random_state = 42)

clf = OneClassSVM(nu=0.1,kernel = "rbf", gamma =0.1)
y_score = clf.fit(X_train).decision_function(X_test)

pred = clf.predict(X_train)

fpr,tpr,thresholds = roc_curve(pred,y_score)

# Plotting roc curve

plt.figure()
plt.plot(fpr, tpr, label='ROC curve (area = %0.2f)' % roc_auc)
plt.plot([0, 1], [0, 1], 'k--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC Curve')
plt.legend(loc="lower right")
plt.show()

The ROC curve I get:

Can somebody help me out with this?

Answer 1

In my opinon, get the scores:

pred_scores = clf.score_samples(X_train)

Then, the pred_scores need to be min-max normalized before min-max normalize

Answer 2

What is bizzare about this plot? You fixed a single set of nu and gamma, thus your model is neither over- nor underfitting. Moving threshold (which is a ROC variable) does not lead to 100% TPR. Try out high gamma and very small nu (which upper bounds the training errors) and you will get more "typical" plots.