CODEWITHSUNDEEP
scikit-learnsvm
Mert Metin
Mert Metin

Reputation: 411

Why SVC, NuSVC and LinearSVC are producing very different results?

I am working on a classification task — geolocation of Twitter users based on their tweets.

I did many experiments by using sklearn's SVC, NuSVC and LinearSVC and bag-of-words model. The accuracies are 35%, 60% and 80%. The difference between SVC and LinearSVC is more than double which is shocking.

I am not quite sure why this is happening exactly. It might be because of overfitting or underfitting? Why is there so much difference between the classifiers?

Upvotes: 0

Views: 1881

Answers (1)

Darius
Darius

Reputation: 12052

In general non-linear kernels are more suitable to model more complex functions than linear functions, but it depends on the data, the chosen hyper parameters (e.g. penalty and kernel) and how you evaluate your results.

LinearSVC

Similar to SVC with parameter kernel=’linear’, but implemented in terms of liblinear rather than libsvm, so it has more flexibility in the choice of penalties and loss functions and should scale better to large numbers of samples.

Source: sklearn.svm.LinearSVC.html#sklearn.svm.LinearSVC

SVC

The implementation is based on libsvm. The fit time complexity is more than quadratic with the number of samples which makes it hard to scale to dataset with more than a couple of 10000 samples.

Source: sklearn.svm.SVC.html#sklearn.svm.SVC

At first you should test a LinearSVC model, because it has just a few hyper parameters and should give you a first result. After that you can try to train a bunch of SVC models and pick the best. For that I recommend to make a gridsearch over C, kernel, degree, gamma, coef0 and tol.

Upvotes: 2

Related Questions