How to detect contiguos images

Question

I am trying to detect when two images correspond to a chunk that matches the other image but there is no overlap.

That is, suppose we have the Lenna image:

Someone unknown to me has split it vertically in two and I must know if both pieces are connected or not (assume that they are independent images or that one is a piece of the other).

A:

B:

The positive part is that I know the order of the pieces, the negative part is that there may be other images and I must know which of them fit or not to join them.

My first idea has been to check if the MAE between the last row of A and the first row B is low.

def mae(a, b):         
    min_mae = 256
    for i in range(-5, 5, 1):
        a_s = np.roll(a, i, axis=1)

        value_mae = np.mean(abs(a_s - b))
        min_mae = min(min_mae, value_mae)
    return min_mae

if mae(im_a[im_a.shape[0] - 1:im_a.shape[0], ...], im_b[0:1, ...]) < threshold:
    # join images a and b

The problem is that it is a not very robust metric.

I have done the same using the horizontal derivative, as well as applying various smoothing filters, but I find myself in the same situation.

Is there a way to solve this problem?

Answer 1

Your method seems like a decent one. Even on visual inspection it looks reasonable:

Top (Bottom row expanded)

Bottom (Top row expanded)

Diff of the images:

It might even be more clear if you also check neighboring columns, but this already looks like the images are similar enough.

Code

import cv2
import numpy as np

# load images
top = cv2.imread("top.png");
bottom = cv2.imread("bottom.png");

# gray
tgray = cv2.cvtColor(top, cv2.COLOR_BGR2GRAY);
bgray = cv2.cvtColor(bottom, cv2.COLOR_BGR2GRAY);

# expand rows
texp = tgray;
bexp = bgray;
trow = np.zeros_like(texp);
brow = np.zeros_like(bexp);
trow[:] = texp[-1, :];
brow[:] = bexp[0, :];
trow = trow[:100, :];
brow = brow[:100, :];

# check absolute difference
ldiff = trow - brow;
rdiff = brow - trow;
diff = np.minimum(ldiff, rdiff);

# show
cv2.imshow("top", trow);
cv2.imshow("bottom", brow);
cv2.imshow("diff", diff);
cv2.waitKey(0);

# save
cv2.imwrite("top_out.png", trow);
cv2.imwrite("bottom_out.png", brow);
cv2.imwrite("diff_out.png", diff);