What is the difference between a decision boundary and a hyperplane?

Question

I am looking into support vector machines and I am wondering what the difference between the decision boundary and the optimal hyperplane is? They both seem to be described as the line drawn to separate the datapoints.

Answer 1

A decision boundary is a hypersurface that partitions the underlying vector space into two sets, one for each class. A general hypersurface in a small dimension space is turned into a hyperplane in a space with much larger dimensions.

Hyperplane and decision boundary are equivalent at small dimension space, 'plane' has the meaning of straight and flat, so it is a line or a plane that separate the data sets. When you do a non-linear operation to map your data to a new feature space, the decision boundary is still a hyperplane in that space, but is not a plane any more at the original space.

Answer 2

The decision boundary for a linear support vector machine is an (affine) hyperplane.

For non-linear kernel support vector machines, the decision boundary of the support vector machine is not an hyperplane in the original feature space but a non-linear hypersurface (a surface of dimension n_features - 1) whose shape depends on the type of kernel.

However, the kernel function can be interpreted as inducing a non-linear mapping from the original feature space to some kernel space. In the kernel space then the decision function of the SVM is an hyperplane. Here is a video that gives an intuitive descriptions of the relation between the two for the polynomial kernel.

Answer 3

When using kernel functions, the decision boundary will no longer remain a straight line.

(It will still be a hyperplane in a different, maybe infinite dimensional space, but this space will not actually be computed or used.)