CODEWITHSUNDEEP
pythonsvm
Willdomybest18
Willdomybest18

Reputation: 137

How to perform multi-class SVM in python

I'm able to understand how to code a binary SVM, for example a simple 1, -1 label. However I am going outside my comfort zone to try and perform multi-class and in effect multi-label SVM. However, I can't find anywhere how to do it.

I am going to use the iris data set, which has three classes. So how do you perform three lables? Is it simply -2, -1 , 1 or 2, 1 , -1?

For example, what differs in the way we train a SVM with two classes then having three. I am trying to implement this from scratch to really get a good understanding rather then just use libraries to get me through.

If anyone can find a good example, or possibly explain the process that would be fantastic. Thanks for your time

Upvotes: 2

Views: 10443

Answers (1)

Ryan S
Ryan S

Reputation: 479

I believe the sklearn library would be helpful here. The sklearn.svm.SVC class is "capable of performing binary and multi-class classification on a dataset.", according to the scikit-learn documentation (https://scikit-learn.org/stable/modules/svm.html).

The labels can actually take any set of integers, as long as they are distinct (e.g. {-1, 1, 2} and {0, 1, 2} and {1, 2, 3} are all valid). Generally, I believe it is best practice to use {0, 1, 2, ..., N} for your label assignments.

Please see the code below for an example:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

N = 1000  # Number of samples

# Create synthetic dataset
X1 = np.random.normal(loc=0, scale=1, size=(N, 2))
Y1 = 0 * np.ones(shape=(1000,))  # LABEL = 0 

X2 = np.random.normal(loc=[-5, 5], scale=1, size=(N, 2))
Y2 = 1 * np.ones(shape=(1000,))  # LABEL = 1

X3 = np.random.normal(loc=[5, -5], scale=1, size=(N, 2))
Y3 = 2 * np.ones(shape=(1000,))  # LABEL = 2

# Create stacked dataset
X = np.vstack((X1, X2, X3))
Y = np.hstack((Y1, Y2, Y3))

# TRAIN SVM LEARNING ALGORITHM
clf = SVC(kernel='linear')
clf = clf.fit(X, Y)

# create decision boundary plot
xx, yy = np.meshgrid(
    np.arange(-10, 10, 0.2),
    np.arange(-10, 10, 0.2))
Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)

# PLOT EVERYTHING
plt.scatter(X1[:,0], X1[:,1], color='r')
plt.scatter(X2[:,0], X2[:,1], color='b')
plt.scatter(X3[:,0], X3[:,1], color='y')
plt.contourf(xx,yy,Z,cmap=plt.cm.coolwarm, alpha=0.8)
plt.title("SVM With Linear Kernel and Three Labels (0, 1, 2)")
plt.show()

enter image description here Hope this helps!

Upvotes: 3

Related Questions