CODEWITHSUNDEEP
opencvfilterpattern-matching
widgg
widgg

Reputation: 1428

pattern recognition between two very different images

So, my problem is that I have to find common points between two images of a microchip. Here's an example of two images: enter image description here enter image description here Between these two images, we can clearly see some common pattern like the wires on the bottom right of the first images that can be found in relatively the same place in the second image. Also, the sort of white Z shape in the first image can be seen in the second images, a bit harder, but it's there.

I tried to match them with SURF (OpenCV), found no common point at all. Tried to apply some filter on both images, like edge detection, thresholding, and other filter that I could found in GIMP, but whatever I tried, no common point were ever found.

I'd like to know if you have any idea to solve this problem ? My suggestion right now would be to manually match key features in both images with line segments, but preferably, it should be automated.

A solution that uses OpenCV would be preferable, but I'm looking for any suggestion possible. In OpenCV, all pattern matching situation that I saw were problems way more obvious that this one. No difference in color and so on.

Upvotes: 3

Views: 2158

Answers (2)

Henrik
Henrik

Reputation: 2229

Unless realtime is required, do a simple approach to test if rotation can be automated:

Circuit boards like the ones in the images, are often based on perpendicular straight line segments. Hence you can "despeckle" and remove stuff like coffee stains, by finding linesegments.

Think about creating a kernel, that have a line with dark pixels on one side, and bright pixels on the other. Fold it on the image (or cross-correlate it) to identify all pixels that have a sequence of bright/dark pixels which are nearly vertical or horizontal.

you may interlace to speed things up. edges of stains and speckles may survive this, if you want angles close to 45* representatations!

The resulting image can be interpreted as a sparse pointcloud. You can now use RANSAC or other similar approaches to describe many of the remaining correlations, as line segments. * use a 2 point line segment as input model for RANSAC, Degrade if small. * Determine infinite lines that have many inliers * use growth or binninng approaches to segmentate lines.

benefits: high likelyhood of line segment representations that are actually present as circuitry in image. 2 point description of segments, possible transforms are easy. easy interpretation of data, as it can be overlayed in openCV

Rotation should be easily found as the rotation that matches most found lines to horizontal and/or vertical axis'es.

  • apply rotation.

repeat for both images.

now you can determine best translation between the images, by simple x,y cross correlation.

Upvotes: 4

user1196549
user1196549

Reputation:

If the top image is always of that quality (quasi bilevel patterns, easy edge detection), I would try a good geometric matching algorithm (such as Cognex or Halcon), training with the top image and searching the bottom one.

Maybe it is worth to first compensate rotation (I hope there is no scaling). You would do that by determining the dominant edge direction, possibly using a Hough transform. Or, much better, by careful mechanical alignment of the sensors.

Anyway, chances of success are low, this is a difficult problem.

Upvotes: 1

Related Questions