Feature Matching Match Rate Between Two Images

Question

I'm making an app which will match the input image with the images from the database.

I'm using this code anyway:

   String path = Environment.getExternalStorageDirectory().getAbsolutePath();  

   Bitmap objectbmp = BitmapFactory.decodeFile(path+"/Sample/Template.jpg");
   Bitmap scenebmp = BitmapFactory.decodeFile(path+"/Sample/Input.jpg");

    Mat object = new Mat(); //from the database
    Mat scene = new Mat(); //user's input image

    // convert bitmap to MAT
    Utils.bitmapToMat(objectbmp, object);
    Utils.bitmapToMat(scenebmp, scene);

    //Feature Detection
    FeatureDetector orbDetector = FeatureDetector.create(FeatureDetector.ORB);
    DescriptorExtractor orbextractor = DescriptorExtractor.create(DescriptorExtractor.ORB);

    MatOfKeyPoint keypoints_object = new MatOfKeyPoint();
    MatOfKeyPoint keypoints_scene = new MatOfKeyPoint();

    Mat descriptors_object = new Mat();
    Mat descriptors_scene = new Mat();

    //Getting the keypoints
    orbDetector.detect( object, keypoints_object );
    orbDetector.detect( scene, keypoints_scene );

    //Compute descriptors
    orbextractor.compute( object, keypoints_object, descriptors_object );
    orbextractor.compute( scene, keypoints_scene, descriptors_scene );

    //Match with Brute Force
    MatOfDMatch matches = new MatOfDMatch();
    DescriptorMatcher matcher;
    matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE);
    matcher.match( descriptors_object, descriptors_scene, matches );

    double max_dist = 0;
    double min_dist = 100;

    List<DMatch> matchesList = matches.toList();

    //-- Quick calculation of max and min distances between keypoints
      for( int i = 0; i < descriptors_object.rows(); i++ )
      { double dist = matchesList.get(i).distance;
        if( dist < min_dist ) min_dist = dist;
        if( dist > max_dist ) max_dist = dist;
      }

     LinkedList<DMatch> good_matches = new LinkedList<DMatch>();

     for( int i = 0; i < descriptors_object.rows(); i++ )
      { if( matchesList.get(i).distance <= 3*min_dist ) 
         { good_matches.addLast( matchesList.get(i));
        }
      }

I am able to produce and count good matches though, what I want is to know the match rate between two matched images like:

Input - Template1 = 35% Input - Template2 = 12% .....................

How to do this?

Answer 1

You could compute a matching rate like goodMatches/totMatches, that is the accuracy of the matching.

Actually there are different ways to do that. The common are:

• Cross check: if T1 matches with T2, check if T2 checks with T1
• Ratio check: as in SIFT, if the best template that matches with T1 is T2, consider the second best match template T2_2 and accept the first match only if the ratio between the matchings is good enough.
• Geometric validation: you should compute the homography between templates and discard the matchings that doesn't agree

I've implemented the first two in Java in an Android application (I used ORB as features).

private List<MatOfDMatch> crossCheck(List<DMatch> matches12, List<DMatch> matches21, List<MatOfDMatch> knn_matches) {

    List<MatOfDMatch> good_matches = new ArrayList<MatOfDMatch>();

    for(int i=0; i<matches12.size(); i++)
    {
        DMatch forward = matches12.get(i);
        DMatch backward = matches21.get(forward.trainIdx);
        if(backward.trainIdx == forward.queryIdx)
            good_matches.add(knn_matches.get(i));   //k=2
    }

    return good_matches;
}



private List<MatOfDMatch> ratioCheck(List<MatOfDMatch> knn_matches, float ratio) {

    List<MatOfDMatch> good_matches = new ArrayList<MatOfDMatch>();

    for(int i=0; i<knn_matches.size(); i++)
    {
        List<DMatch> subList = knn_matches.get(i).toList();

        if(subList.size()>=2)
        {
            Float first_distance = subList.get(0).distance;
            Float second_distance = subList.get(1).distance;

            if((first_distance/second_distance) <= ratio)
                good_matches.add(knn_matches.get(i));


        }

    }

    return good_matches;
}