CODEWITHSUNDEEP
machine-learningclassificationvowpalwabbit
AmirC
AmirC

Reputation: 326

Vowpal Wabbit Multiclass Linear Classification

Is it possible to train a multiclass (multinomial) linear classification model with Vowpal Wabbit Library?

I tried to use --oaa with --loss_function squared, but it seems that the default loss function for --oaa is logistic.

I am using rcv1.multiclass as input.

One Solution:

I can create multiple version of the data as follows:

Version i: make all the labels zero except class I

Then I can train multiple binary classifications for each version of data. Finally, I can feed the test data to all the classifier and apply an argmax. Is there any better (automated) solution?

Upvotes: 1

Views: 750

Answers (1)

Martin Popel
Martin Popel

Reputation: 2670

When you use vw --oaa N, you will actually get a linear N-class classifier. To get a non-linear classifier you would need to add quadratic/polynomial features (-q, --cubic, --interactions) or kernels (--ksvm) or a hidden layer (--nn) or any other nonlinear reduction (--lrq, --stage_poly, --autolink).

The choice of loss function does not effect whether the classifier is linear or not. The default is --loss_function=squared. For classification, I would suggest to use --loss_function=logistic (possibly with --probabilities if you want to predict probability of each class) or --loss_function=hinge (if you care only about the top class).

Then I can train multiple binary classifications for each version of data. Finally, I can feed the test data to all the classifier and apply an argmax. Is there any better (automated) solution?

Yes, this is exactly what --oaa does (but more efficiently).

Upvotes: 2

Related Questions