Can I use face landmarks returned from microsoft face API to track the face in real time

Question

My target is to use Microsoft face API cognitive service to detect the faces in a frame , then using the landmarks returned for each face I would track it using optical flow for example!.

My question is about the accuracy .. is this approach would work properly , or there are some other logical constrains exists behind tracking face using its landmarks?

Answer 1

A short answer is Yes.

Actually, Microsoft Cognitive Services - Video API [1] already support face tracking. In case you want to handle it by yourself, Face API use almost the same technique for implementation.

For tracking, optical flow should be enough, we also have demo that judge same person by the overlap between the detected rectangles. You can also invoke the verification API [2] for a more precise result.

But there are always corner cases, everything should work fine when the face is detectable (cases like side face is still challenging) and not moving too fast.

If you have any further questions, feel free to update.

• [1] https://www.microsoft.com/cognitive-services/en-us/video-api
• [2] https://westus.dev.cognitive.microsoft.com/docs/services/563879b61984550e40cbbe8d/operations/563879b61984550f3039523a