CODEWITHSUNDEEP
kernelscikit-learnsvm
Akash Kumar singh
Akash Kumar singh

Reputation: 49

Classification through Radial Basis Function (RBF) SVM

I am using sklearn.svm.SVC (kernel='rbf') for the classification of an image data, which is doing pretty well job. Linear SVM classifies the data by putting a hyper plane between the two classes. In the case of rbf SVM the plane would be in infinite dimension. For any testing point we can use predict to check which it belongs to. In linear case we can manually get the prediction by getting the equation of the hyper plane. How can we do this in rbf SVM case. How exactly predict works in rbf SVM case.

Upvotes: 3

Views: 5272

Answers (1)

edgarmtze
edgarmtze

Reputation: 25048

Fisrt things first

Whenever we classify we should consider:

  • Classifiers can be learnt for high dimensional features spaces, without actually having to map the points into the high dimensional space.
  • Data may be linearly separable in the high dimensional space, but not linearly separable in the original feature space
  • Kernels can be used for an SVM because of the scalar product in the dual form, but can also be used elsewhere – they are not tied to the SVM formalism.
  • Kernels apply also to objects that are not vectors enter image description here

For instance I will put some used Kernels. enter image description here

For a SVM Classifier with Gaussian Kernel we would have something like:

enter image description here As you notice support vector is substituted and therefore we could vary it depending on results, for example, consider two features and their colored points: enter image description here

And setting some values we get: enter image description here

Now enter image description here

Or enter image description here

Now what happens when infinity comes to play:

enter image description here

Then: enter image description here enter image description here

And what about adaBoost to play with datasets http://cseweb.ucsd.edu/~yfreund/adaboost/

If you like you could test The NETLAB ML Matlab software by Ian Nabney here

Here are more sources for SVM

  • Christopher M. Bishop, "Pattern Recognition and Machine Learning" , Springer (2006), ISBN 0-38-731073-8.
  • Hastie, Tibshirani, Friedman, "Elements of Statistical Learning", Second Edition, Springer, 2009. Pdf available online.
  • Ian H. Witten and Eibe Frank, "Data Mining: Practical Machine Learning Tools and Techniques" , Second Edition, 2005.
  • David MacKay, "Information Theory, Inference, and Learning Algorithms" Which is freely available online!
  • Tom Mitchell, "Machine Learning" , McGraw Hill, 1997

Upvotes: 8

Related Questions