Classify faces from VNFaceObservation

Question

I'm working with Vision framework to detect faces and objects on multiple images and works fantastic.

But I have a question that I can't find on documentation. The Photos app on iOS classify faces and you can click on face and show all the images with this face.

How can I classify faces like Photos app? Is there any unique identifier or similar to do this?

Thanks!

Answer 1

In order to uniquely recognise faces, firstly you need to detect a face, then run it through a CoreML model (or another image classification model type, such as a Tensorflow model) in order to classify the image and tell you the likeliness that the face you captured matches one of the faces trained into your model.

Apple Photos uses machine learning (as mentioned in their iPhone reveal keynote this year) to train the device to recognise faces in your photos. The training would be performed locally on the device, however, Apple does not offer any public APIs (yet) to allow us to do this.

You could send photo data (crops of faces using the tool mentioned above by Paras) to your server and have it train a model (using CoreML's trainer or something like Nvidia DIGITS on AWS or your own server), convert it to CoreML, compile the model then download it to your device and sideload the model. This is as close as you're going to get to "magic" face recognition used by Photos, for now, as the device can only read compiled models.

Answer 2

I don't think their is a way to uniquely identify faces returned to you from the vision framework. I checked the UUID property a VNFaceObservation and it is a different identifier every-time.

You might have to make your own CoreML model or just wait/find a good 3rd party one.

I hope someone proves me wrong because I want to know also.

Answer 3

You might want to check out this repo https://github.com/KimDarren/FaceCropper

I tested this and works very well, you can even customise as per your need.