CODEWITHSUNDEEP
pythonopencvsift
user1371666
user1371666

Reputation: 497

using SIFT to locate correspondence between images


I have a 512x512 image which gave 6109 SIFT keypoints. Now, I rotated it then I got 6070 SIFT keypoints.

I tried following

import cv2
import numpy as np
np.set_printoptions(threshold=np.nan)
img = cv2.imread('home.jpg')
gray= cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
sift = cv2.xfeatures2d.SIFT_create()
kp,des = sift.detectAndCompute(gray,None)
print('kp',len(kp))
print('des',des.shape)
for i in range(len(kp)):
    print('x ',kp[i].pt[0])
    print('y ',kp[i].pt[1])
    print('size',kp[i].size)
    print('response',kp[i].response)
    print('descriptor',np.sort(des[i]))
    print('\n')

Send the output to a text file.
Now I did the same with rotated image.
You must have noticed that I am sorting the 128 value descriptor before writing it in file so that on rotation if order of values changes in descriptor then still there will be match.

My conclusion is that among those 12179 descriptors(6070+6109), only two were identical. Rest differed atleast by one number,usually by many I saw.
Can you suggest a python code that will show the x,y coordinates of correspondences between the two images ?
Also, there is something fundamentally wrong i must be doing here in comparing descriptors.Please point that out .
Thanks

Upvotes: 0

Views: 2150

Answers (2)

user1371666
user1371666

Reputation: 497

this is implementation of details provided by Garvita Tiwari.
I am using python 2.7.12 with opencv 3.4.1

import cv2
import numpy as np
src_img = cv2.imread('src.jpg')
test_img = cv2.imread('test.jpg')
src_gray= cv2.cvtColor(src_img,cv2.COLOR_BGR2GRAY)
test_gray= cv2.cvtColor(test_img,cv2.COLOR_BGR2GRAY)
sift = cv2.xfeatures2d.SIFT_create()
src_kp,src_desc = sift.detectAndCompute(src_gray,None)
test_kp,test_desc = sift.detectAndCompute(test_gray,None)
bf = cv2.BFMatcher()
matches = bf.knnMatch(src_desc,test_desc, k=2)
good = []
for m,n in matches:
    if m.distance < 0.75*n.distance:
        good.append([m])
print 'so the correspondences are'
for i in range(len(good)):
    print int(src_kp[good[i][0].queryIdx].pt[0]),int(src_kp[good[i][0].queryIdx].pt[1]) ,'->',int(test_kp[good[i][0].trainIdx].pt[0]) ,int(test_kp[good[i][0].trainIdx].pt[1])

Upvotes: 1

Garvita Tiwari
Garvita Tiwari

Reputation: 584

  1. Why are you sorting the descriptor? 128-dimensional descriptor is something that describes the feature keypoint(as the name suggests). By sorting you are essentially losing the description of your key points/features.

  2. Searching for an identical descriptor is not a good way to find correspondence. You have to find descriptors with minimum Euclidean distance(or some other distance measure), to get correspondence. There are many algorithms for that in opencv like Brute force matches or knn based matcher or flann based matcher. YOu have to give the descriptor as input to these matcher(do not modify your decsriptor values by sorting or some other opertation) For example

    kp1, des1 = sift.detectAndCompute(img1,None)
kp2, des2 = sift.detectAndCompute(img2,None)

# BFMatcher with default params
bf = cv2.BFMatcher()
matches = bf.knnMatch(des1,des2, k=2)

matches will have index for correspondences and distance between those correspondences

You can get sort this matches based on matches.distance and get x and y coordinate using matches.queryIdx and matches.trainIdx

you can get x and y location using kp1[matches[idx].trainIdx] and kp2[matches[idx].queryIdx]

Upvotes: 2

Related Questions