CODEWITHSUNDEEP
matlabregistration
SimaGuanxing
SimaGuanxing

Reputation: 691

Matlab Automated Feature Matching Between Two Images in Different Resolution

I am currently working on a multi-modal registration problem for images with different resolution. For the images with relative same resolution, I can match my target (cubic object in the middle) well by using Matlab multi-modal registration pipeline.

However, if the there are large scale changes between moving and fixed images, the intensity based multi-modal registration doesn't work.

I tried to use sift/surf feature matching to recover scale but since the two images are from very different device, I was not able to get correct results. I have been thinking about deep learning approach but not sure how to get start. Any ideas or suggestions are welcome.

I have attached my code below so you can feel free to test it in your end.

moving = imread('moving.png');
fixed = imread('fixed1.png');

figure
subplot(311)
imshowpair(fixed,moving,'montage')
title('Before Registration')

 % Optimizer and Metric setting
optimizer = registration.optimizer.OnePlusOneEvolutionary;
optimizer.GrowthFactor = 1.05; %1.0001
optimizer.Epsilon = 1.5e-06;
optimizer.InitialRadius = 0.00625; %0.0022 0.0015
optimizer.MaximumIterations = 200; %500


metric = registration.metric.MattesMutualInformation;
metric.NumberOfSpatialSamples = 500;
metric.NumberOfHistogramBins = 25; 


tformRigid = affine2d([1 0 0;0 1 0; 0 0 1]);
[movingRegistered,~,tform] = imregister2(moving,fixed,'affine',optimizer,metric,'DisplayOptimization',false);


subplot(312)
imshowpair(fixed,movingRegistered,'montage')
title('After Rigid Transform (Fix Spatial Difference)')

Upvotes: 1

Views: 336

Answers (1)

Soleil
Soleil

Reputation: 7459

I see a parallelepipedic-ish object, rather than cubic. I would stick to classical CV rather than DL, and work with geometric computer vision.

What you're trying to do is to find a transformation (function) that would make the two objects invariants (ie., they are the same). In CV you can work with different aspects of the image (color, intensities, gradients, mipmaps, etc); what I see that is common ie., that will help you find your function, is the shape (2D and 3D), the geometry of your objects.

I would advise you to try few of the algorithms from geometric computer vision [1] and review geodesic methods [2], [3], the latter will allow you to deal with different volumetric shapes, and not just a certain flat shape (eg., rectangle with ratio:=width/height). I would go for these steps:

  • prepare the images to make them as similar (actually make their space of expression as similar as possible) as possible (dimension reduction; eg., normalize contrast and luminosity, add a lowpass filter, detect border)
  • and then go for learning with a classifier (SVM, AdaBoost ...); you'll need to prepare your groundtruth, learning set and test set (there are tons of references for that).

Registration might be useful for the preparation step; it depends on your pipeline.

What goes with DL is that everything is built at once (it's an inconvenient if you look for modularity, it's and advantage if you want an "easier" preparation), but it also requires really big sets and it's very computing intensive (need time and computing power). Depending on your requirements (time, money, quality of results), you might find one method to fit better to your project, but you migth want to try another or all of them (like a benchmark/review).

[1] Elements of Geometric Computer Vision, Andrea Fusiello http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO4/tutorial.html

[2] Geodesic Methods in Computer Vision and Graphics, Gabriel Peyré, Mickael Péchaud, Renaud Keriven and Laurent D. Cohen https://www.researchgate.net/publication/47523356_Geodesic_Methods_in_Computer_Vision_and_Graphics

[3] https://scholar.google.com/scholar?hl=en&num=100&ie=UTF-8&q=computer+vision+geodesic

Upvotes: 1

Related Questions