How set miscalculation cost in MATLAB SVM model?

Question

There is a option in MATLAB R2015b fitcsvm to set miscalculation cost in Support vector machine(SVM) model. This is the documentation:

'Cost' — Misclassification cost square matrix | structure array Misclassification cost, specified as the comma-separated pair consisting of 'Cost' and a square matrix or structure. If you specify:

The square matrix Cost, then Cost(i,j) is the cost of classifying a point into class j if its true class is i (i.e., the rows correspond to the true class and the columns correspond to the predicted class). To specify the class order for the corresponding rows and columns of Cost, additionally specify the ClassNames name-value pair argument. The structure S, then it must have two fields: S.ClassNames, which contains the class names as a variable of the same data type as Y S.ClassificationCosts, which contains the cost matrix with rows and columns ordered as in S.ClassNames For two-class learning, if you specify a cost matrix, then the software updates the prior probabilities by incorporating the penalties described in the cost matrix. Subsequently, the cost matrix resets to the default. For more details on the relationships and algorithmic behavior of BoxConstraint, Cost, Prior, Standardize, and Weights, see Algorithms.

The defaults are:

For one-class learning, Cost = 0. For two-class learning, Cost(i,j) = 1 if i ~= j, and Cost(i,j) = 0 if i = j. Example: 'Cost',[0,1;2,0]

Data Types: double | single | struct

• What are effects of this option in SVM? Is this similar to combine sensitivity and specificity and weight more one of those comparing to another?

• What is clear meaning of 1 and 2 in example ([0,1;2,0])?

Answer 1

What are effects of this option in SVM? Is this similar to combine sensitivity and specificity and weight more one of those comparing to another?

Yes, these are just weights used for focusing more on a particular class. In SVM this is obtained through penalizing missclassification of a specified class more, or equivalently - changing limit of a corresponding lagrange multiplier in the dual optimziation process.

What is clear meaning of 1 and 2 in example ([0,1;2,0])?

This matrix is of form

0 1
2 0

thus it means:

• if point has class 1 and we assign it class 1, penalty is 0 (correct classification)
• if point has class 1 and we assign it class 2, penalty is 1
• if point has class 2 and we assign it class 1, penalty is 2
• if point has class 2 and we assign it class 2, penalty is 0 (correct classification)

Consequently we care twice as much for correct classification of class 2 than for class 1. In SVM this would mean that the cost is of form

1/2 ||w||^2 + C SUM_{i=1}^N xi_i cost_i

and cost_i = 1 if class of i'th sample is 1, and cost_i =2 otherwise. As you can see - simply we add weights to the missclassification term in the SVM cost.