CODEWITHSUNDEEP
matlabclassificationsvm
user1943584
user1943584

Reputation: 67

why as number of features increases, the classification accuracy decreases when using svm

I am using libsvm for image classification. Why when I use more features for classification my prediction accuracy decreases? Shouldn´t it increase? My dataset size is fixed at 1600 for training and 400 for testing.

Upvotes: 2

Views: 2163

Answers (1)

Will Faithfull
Will Faithfull

Reputation: 1908

Because the additional features may not be at all useful for separating the classes in the feature space. Accuracy is not necessarily tied to number of features.

Including lots of poor features may cause your SVM to learn the noise in the data, damaging the accuracy.

For example, if your extra feature looks like this (using 2D plots for clarity):

class1 = red, class2 = blue

Then it will not be a very good feature for separating the (in this case) two classes. If for example, the SVM trains only on this pattern, it will not be very good at predicting the class of a future point. However, there might be a feature in your dataset that looks like this: class1 = red, class2 = blue

A feature like this would be very useful in separating the two classes.

Upvotes: 6

Related Questions