CODEWITHSUNDEEP
machine-learningcomputer-visiondata-scienceface-recognition
Pushpa
Pushpa

Reputation: 942

How to calculate LFW accuracy of a face recognition model?

In my research I have observed many of the face recogntion algorithms propose their model accuracy interms of LFW dataset accuracy. I see that LFW dataset has images of 5749 different people and there is no split of training and testing.

I have developed my own DNN model implemented for face recognition which is similar to facenet architecture. Can anyone help with the follwoing questions?

1) Which specific accuracy parameter on LFW everyone is talking about? I knew the accuracy parameters such as ROC, precision, recall and FAR vs FRR plot. Is it something related to these parameters?

and

2) The procedure to calculate LFW accurcay of my model? [Any open source link is appreciable]

Upvotes: 2

Views: 4005

Answers (2)

sefiks
sefiks

Reputation: 1650

LFW is already splitted into train, test and 10_folds in scikit-learn.

from sklearn.datasets import fetch_lfw_pairs
lfw_pairs_test = fetch_lfw_pairs(subset = 'test')
pairs = lfw_pairs_test.pairs
target = fetch_lfw_pairs.target

Pairs store 2 image items. You can test these pairs in your custom model and compare the prediction with target value.

Upvotes: 2

fytao
fytao

Reputation: 331

I see that LFW dataset has images of 5749 different people and there is no split of training and testing.

Actually LFW offers splits for train and test. Details in http://vis-www.cs.umass.edu/lfw/#views.

Which specific accuracy parameter on LFW everyone is talking about? I knew the accuracy parameters such as ROC, precision, recall and FAR vs FRR plot. Is it something related to these parameters?

LFW adopts ROC curve (the figures), mean classification accuracy u and standard error of the mean S_E (the tables). The ROC curve used by LFW is plot with FPR (False Positive Rate) and TPR (True Positive Rate).

The procedure to calculate LFW accurcay of my model?

Two useful links:

  1. The definition of FPR, TPR, and TP, FN, TP, TN: https://en.wikipedia.org/wiki/Confusion_matrix

  2. There is a wiki page in the Facenet github repo, where you can read the source code to learn how to draw the ROC curve and compute u and S_E: https://github.com/davidsandberg/facenet/wiki/Validate-on-LFW

Procedure to get the ROC curve and u and S_E:

  1. We need pairs.txt from the testing section of LFW, which contains 10 sets of 300 matched and 300 mismatched pairs respectively. So 6,000 pairs in total, half of which are matched, the other half of which are mismatched.

  2. Set thresholds range from 0 to 1, gap 0.001 for example, which produces 1,000 thresholds.

  3. Calculate the distance d over all pairs. For a threshold t, if d <= t, pairs are predicted as matched. Otherwise, pairs are predicted as mismatched.

  4. For each threshold t:

    1. For each matched pair,

      1. If it is predicted as matched, TP+=1;

      2. If it is predicted as mismatched, FN+=1.

    2. For each mismatched pair,

      1. If it is predicted as matched, FP+=1;

      2. If it is predicted as mismatched, TN+=1.

    3. Compute TPR, FPR, Acc:

      1. TPR = TP / (TP + FN);

      2. FPR = FP / (FP + TN);

      3. Acc = (TP + TN) / (TP + FN + FP + TN).

  5. We get a 1,000-length list like [(TPR_t1, FPR_t1, Acc_t1), (TPR_t2, FPR_t2, Acc_t2), ...]. The ROC curve can be produced by plotting FPRs against TPRs. u is the mean of Accs, while S_E is the standard deviation of Accs.

Upvotes: 3

Related Questions