iOS Facial Recognition Continuous Learning

Question

I was tasked to find the best way to create a facial recognition feature on an app with machine learning. This feature will be used to clock employees into the app. The feature will support...

• multiple users per device.

• continuous training (so when the mlmodel recognizes someone, it will send new images to the model on the back-end and train the model with the new recently taken images)

• updates new classes (when a new user comes along and wants to use the app, the app will take pictures of them, send those images to the model training program on the back-end which will train the mlmodel to recognize the new user)

  • sends the newly updated models to other devices in the same store so they will recognize the employees as well

What I've tried.

• I've tinkered with on-device training and Knn. But from what I understand on-device training will not work for this, because the on-device training models can only have up to 10 classes and knn isn't giving very accurate results...at all

• Manual training and re-training with the createML. This is when I...

  1. train a model with createML on my mac
  2. download the model to the app with URLSession
  3. add a new user with the app or take updated pictures of old users
  4. send the images of the new user/updated old user to createML on my mac
  5. create a whole new model with all the images I've ever taken of all the users
  6. repeat steps 2-5 forever

  This works just fine but is unbelievably expensive, time-consuming, and unfeasible for the number of eventual users the app will have, to do over and over again.

  I'm still very new to machine learning and I feel like I'm going about this the wrong way. I would just like to see if anyone knows of a better/more efficient method of continuous learning so the model will remember what it's learned previously and I can just add new classes or images to it with createML... or if someone could point me in the right direction.

Answer 1

Take a look at Turi Create -- also from Apple: https://github.com/apple/turicreate

It can do everything Create ML does, but is in python and programmable, so you could automate the whole process on your backend. If you know how to do it in CreateML, you will find Turi Create easy to pick up.

Answer 2

To have an accurate result you should look into more powerful machine learning models. Here is an example of a really powerful face recognition model: https://github.com/davidsandberg/facenet.

Now, the next question becomes how would you integrate your app with this new model. This is really up to you but I would recommend you to checkout a few backend alternatives like leveraging AWS Services (EC2 compute servers , Sagemaker, API Gateway, etc) to run and coordinate the inferences. A couple of benefits to doing this is that your app would just be mainly front-end thus making it light and also scalable across different and older IOS platforms and devices. But more importantly, it gives you extra leg-space to do more sophisticated things in the future, where as using CoreML you will be mainly limited to on-device computational power and also the swift-based language.

However, leveraging cloud services would also have other cons attached like the learning curve (learning AWS Services) and potentially privacy issues.

This is just one of the ways, there are many other similar cloud providers like Google , IBM and Azure. Without knowing further your timeline, budget, technical expertise, I can only give you these options and the rest of the choice is yours to make