CODEWITHSUNDEEP
machine-learningclassificationsvm
smwikipedia
smwikipedia

Reputation: 64381

What does the "support" mean in Support Vector Machine?

What the meaning of the word "support" in the context of Support Vector Machine, which is a supervised learning model?

Upvotes: 4

Views: 2730

Answers (3)

Dimitrios Bouzas
Dimitrios Bouzas

Reputation: 42929

In SVMs the resulting separating hyper-plane is attributed to a sub-set of data feature vectors (i.e., the ones that their associated Lagrange multipliers are greater than 0). These feature vectors were named support vectors because intuitively you could say that they "support" the separating hyper-plane or you could say that for the separating hyper-plane the support vectors play the same role as the pillars to a building.

Now formally, paraphrasing Bernhard Schoelkopf 's and Alexander J. Smola's book titled "Learning with Kernels" page 6:

"In the searching process of the unique optimal hyper-plane we consider hyper-planes with normal vectors w that can be represented as general linear combinations (i.e., with non-uniform coefficients) of the training patterns. For instance, we might want to remove the influence of patterns that are very far away from the decision boundary, either since we expect that they will not improve the generalization error of the decision function, or since we would like to reduce computational cost of evaluating the decision function. The hyper-plane will then only depend on a sub-set of the training patterns called Support Vectors."

That is, the separating hyper-plane depends on those training data feature vectors, they influence it, it's based on them, consequently they support it.

Upvotes: 4

Has QUIT--Anony-Mousse
Has QUIT--Anony-Mousse

Reputation: 77475

In a kernel space, the simplest way to represent the separating hyperplane is by the distance to data instances. These data instances are called "support vectors".

The kernel space could be infinite. But as long as you can compute the kernel similarity to the support vectors, you can test which side of the hyperplane an object is, without actually knowing what this infinite dimensional hyperplane looks like.

In 2d, you could of course just produce an equation for the hyperplane. But this doesn't yield any actual benefits, except for understanding the SVM.

Upvotes: 1

Simo Erkinheimo
Simo Erkinheimo

Reputation: 1397

Copy-pasted from Wikipedia:

Maximum-margin hyperplane and margins for an SVM trained with samples from two classes. Samples on the margin are called the support vectors.

Upvotes: 5

Related Questions