CODEWITHSUNDEEP
pythonmachine-learningscikit-learnsvm
AjitKrish
AjitKrish

Reputation: 13

Predicting Class in Multiclass classification using SVM

In:svc1.predict([X_poly[25].reshape(1,-1))

Out:array([3], dtype=int64)

In:svc1.decision_function([X_poly[25]])

Out:array([[ 0.58348329, -0.99979492,  0.08571431, -0.99996706, -0.52397444,
     0.99959056]])

In:svc1.predict_proba(X_poly[25].reshape(1,-1))

Out:array([[ 0.48373954,  0.11870606,  0.00261101,  0.39494339]])

I am using rbf kernel and 'ovo' classification. I am trying to predict the class of 26th sample in the data. The SVC classifier predicted the class sample as 3. However, I am not able to comprehend how does it narrow down to class 3 when probabilities suggest the other way. Can you also please advise the interpretation of the output of decision_function method and how it is used to predict the class.

Upvotes: 0

Views: 1737

Answers (1)

O.Suleiman
O.Suleiman

Reputation: 918

As the documentation suggests:

The probability model is created using cross validation, so the results can be slightly different than those obtained by predict. Also, it will produce meaningless results on very small datasets.

I think this is why you have different outputs.

As for the ovo (one-vs-one) decision function, since your decision function returns 6 values that means you have 4 classes:

(n*(n-1))/2 = 6, where n is the number of classes.

Now for how to predict the class using the numbers returned from the decision function. As the name suggests (one-vs-one), it compares the classes one-to-one and then outputs a positive or a negative number on which class it thinks is the right one. In your case, and since we have 4 classes, it produces 6 numbers which represent the comparison between your classes (1,2,3,4) as follows: [12,13,14,23,24,34] which is represented in your case by this decision vector: 

[0.58348329, -0.99979492,  0.08571431, -0.99996706, -0.52397444, 0.99959056]

So, which class wins depends on the signal of the numbers, so in this example, the winner classes are: [1,3,1,3,4,3], i.e. '12' is compared by 0.58348329, hence class '1' wins ... etc. Therefore, the decision function votes the class '3' as your prediction since it is the most winning class.

Upvotes: 1

Related Questions