CODEWITHSUNDEEP
machine-learningcomputer-visionsvm
zardav
zardav

Reputation: 1248

SVM - Can I normalize W vector?

In SVM, there is something wrong with normalize W vector such:
for each i W_i = W_i / norm(W)
I confused. At first sight it seems that the result sign(<W, x>) will be same. But if so, in the loss function norm(W)^2 + C*Sum(hinge_loss) we can minimize W just by do W = W / (large number).
So, where am I wrong?

Upvotes: 1

Views: 1380

Answers (2)

Martin Thoma
Martin Thoma

Reputation: 136347

I suggest you to read either my minimal 5 ideas of SVMs or better

[Bur98] C. J. Burges, “A tutorial on support vector machines for pattern recognition”, Data mining and knowledge discovery, vol. 2, no. 2, pp. 121–167, 1998.

To answer your question: SVMs define a hyperplane to seperate data. Hyperplanes are defined by a normal vector w and a bias b:

enter image description here

If you change only w, this will give another hyperplane. However, SVMs do more tricks (see my 5 ideas) and the weight vector actually is normalized to be in a relationship to the margin between the two classes.

Upvotes: 2

Amitoz Dandiana
Amitoz Dandiana

Reputation: 150

I think you are missing out on the constraint that r(wTx+w0)>=1 for all examples, thus normalizing the weight vector will violate this constraint.

In fact this constraint is introduced in the SVM in the first place to actually achieve a unique solution like else you mentioned there are infinite solutions possible just by scaling the weight vector.

Upvotes: 1

Related Questions