Optimized way of face recognition using Azure Face API

Question

I need to implement face recognition using azure face api . I have developed a programme which is able to find similiar faces using .net SDK . For my use case ,I need to click photo of a person from the webcam and find matching faces from images kept in azure cloud storage . Now, there could be thousand of images in azure cloud storage and in my current implementation of face recognition ,I'm iterating through all the images(kept in azure cloud storage ) and then matching them with the webcam image . The concern here is : The face api (provided by azure ) charges 1 dollar per thousand call . Is there a way the search could be optimized such that i don't have to scan the faces which i have already scanned for previous searches

public async Task<List<DetectedFaceAttributes>> FindSimiliarFacesWithAttributesFromContainer(IFaceClient client, string RECOGNITION_MODEL1, string sourceImageFileName)
        {
            string url = BlobBaseURL;
            string sourceurl = sourceContainerURL;
            var imagesInNovotraxContainer = await _blobService.GetNames();
            IList<Guid?> targetFaceIds = new List<Guid?>();
            var faceList = new List<DetectedFaceAttributes>();

            // Detect faces from source image url.
            IList<DetectedFace> detectedFaces = await DetectFaceRecognize(client, $"{sourceurl}{sourceImageFileName}", RECOGNITION_MODEL1);
            if (detectedFaces.Any())
            {
                foreach (var targetImageFileName in imagesInNovotraxContainer)
                {
                    var faceattribute = new DetectedFaceAttributes();
                    // Detect faces from target image url.
                    var faces = await DetectFaceRecognizeWithAttributes(client, $"{url}{targetImageFileName}");
                    // Add detected faceId to list of GUIDs.
                    if (faces.Any())
                    {
                        targetFaceIds.Add(faces[0].FaceId.Value);
                        faceattribute.DetectedFace = faces[0];
                        faceattribute.ImageFileName = targetImageFileName;
                        faceList.Add(faceattribute);
                    }
                }


                // Find a similar face(s) in the list of IDs. Comapring only the first in list for testing purposes.

                IList<SimilarFace> similarResults = await client.Face.FindSimilarAsync(detectedFaces[0].FaceId.Value, null, null, targetFaceIds);

                var similiarFaceIDs = similarResults.Select(y => y.FaceId).ToList();
                var returnDataTypefaceList = faceList.Where(x => similiarFaceIDs.Contains(x.DetectedFace.FaceId.Value)).ToList();
                return returnDataTypefaceList;
            }
            else
            {
                throw new Exception("no face detected in  captured photo ");
            }

public async Task<List<DetectedFace>> DetectFaceRecognize(IFaceClient faceClient, string url, string RECOGNITION_MODEL1)
        {
            // Detect faces from image URL. Since only recognizing, use the recognition model 1.
            IList<DetectedFace> detectedFaces = await faceClient.Face.DetectWithUrlAsync(url, recognitionModel: RECOGNITION_MODEL1);
            //if (detectedFaces.Any())
            //{
            //    Console.WriteLine($"{detectedFaces.Count} face(s) detected from image `{Path.GetFileName(url)}` with ID : {detectedFaces.First().FaceId}");
            //}
            return detectedFaces.ToList();
        }

Answer 1

Your implementation is not totally clear for me in terms of calls to Face API / your storage (what's behind "DetectFaceRecognizeWithAttributes"). But I think you are right in the fact that you missed something and your global processing is over costly.

What you should do depends on your target:

• Is it face "identification"?
• Or face "similarity"?

Both have the same logic, but they are using different API operations

Case 1 - Face identification

Process

The global process is the following: you will use a "Person Group" or "Large Person Group" (depending of the number of persons you have) to store data about faces that you already know (the one in you storage), and you will use this group to "identify" a new face. with that, you will do "1-n" search, not "1-1" as you do right now.

Initial setup (group creation):

Choose if you need Person Group or Large Person group, here are the actual limits depending on your pricing:

• Person Group:
  • Free-tier subscription quota: 1,000 person groups. Each holds up to 1,000 persons.
  • S0-tier subscription quota: 1,000,000 person groups. Each holds up to 10,000 persons.
• Large Person Group:
  • It can hold up to 1,000,000 persons.
  • Free-tier subscription quota: 1,000 large person groups.
  • S0-tier subscription quota: 1,000,000 large person groups.

Here I am using Person Group in the explanation, but it's the same methods.

When you know the one you need, create it using "Create" operation. Then, for each person, you will have to create a "PersonGroup Person" using "PersonGroup Person - Create", and add the corresponding faces to it using "PersonGroup Person - Add Face". Once it is done, you never need to reprocess "detect" operation on those faces.

Then for the "run" part

When you have a new image that you want to compare:

1. Detect faces in your image with Detect endpoint of Face API
2. Get the face Ids of your result
3. Call Identify endpoint of Face API to try to identify those face Ids with your (large) person group

To limit the number of call, you can even do batches of identification calls (up to 10 "input" face Ids in 1 call - see doc).

Case 2 - Face similarity

Here you can use a "Face List" or "Large Face List" to store the faces that you already know, and pass the id of this list when calling "Find Similar" operation. Example with FaceList:

1. Start with "FaceList - Create" to create your list (doc)
2. Use "FaceList - Add Face" to add all the faces that you have currently in your blob (doc)
3. Then for the run, when you call "Find Similar", provide the ID of your FaceList in "faceListId" parameter and the id of the face you want to compare (from Face Detect call)