CODEWITHSUNDEEP
machine-learning
JTCrafter
JTCrafter

Reputation: 9

How to deal with label that not included in training set when doing prediction

For example, using supervise learning to classify 5 different people face. But when test on 6th people face that not in training set, the model will still predict it within the 5 people. How to let the model predict the 6th and onwards people face as unknown when the model doesn't train them before?

Upvotes: -2

Views: 1813

Answers (2)

C. Arnold
C. Arnold

Reputation: 89

The best way to approach this is to create a model for each known image and have it predict the accuracy for each one separately (binary selection for each one). For instance, for image 3 all the images that correspond to it should make the model output 1, and for the images that don't it should return 0. Then you combine all the models and check if all of them returned accuracy above a certain threshold, e.g. 0.8.

Unfortunately, even though what Anant Mittal mentioned in his answer makes sense, it would not work. I tried it. Because if the image is unknown, the model would compare it to the other ones and check from which ones it differs most. If for instance you have 5 images, and the unknown image differs more from images 1-4 than it is from image 5, then the accuracy it will show for images 1-4 will be low (say p1, p2, p3, and p4 respectively) but then the accuracy for image 5 would be 1-p1-p2-p3-p4. Meaning, it can be very high if the image does not resemble the initial 4 images.

Upvotes: 0

Anant Mittal
Anant Mittal

Reputation: 2133

You could set a certain threshold for prediction the known classes. Your model should predict from the known classes only if it predicts it with a certain threshold value, otherwise, it will be classified as unknown.

The other (and less preferable) way to deal with this problem is to have another class called unknown even during training and put some random faces as corresponding examples of this class.

Upvotes: 1

Related Questions