Why is openCV warpAffine setting my entire image to zero?

Question

So, I'm trying to write a function in OpenCV Python that will take an image, a binary image showing identified finger region, and a 2x(image width) matrix that contains location data for the edges of a finger that has been found in that image as input, and then normalise the image and return that as output.

To normalise, I'm projecting a baseline through the finger region parallel to the edges, then trying to turn the image so that that baseline is parallel to the edges of the image rectangle. So the function draws that baseline, determines an affine transform matrix and then uses warpAffine to normalise the image. Unfortunately, for reasons I am very unclear on, the output of warpAffine is just a matrix full of zeroes. My code is shown below. Any help would be greatly appreciated, thank you! :)

def huang_normalise(img, fvr, edges):
 # get image size
 img_h, img_w = img.shape
 fvr_h, fvr_w = fvr.shape

 # get a baseline
 bl = (edges[0, :edges.shape[1]] + edges[1, :edges.shape[1]])/2

 # fit a straight line to the baseline
 brob = np.polyfit(range(0, img_w), bl, 1)
 rot = -1*math.atan(brob[1])
 tr = img_h/2 - brob[0]


 # affine transform
 rotmat = np.float32([[math.cos(rot), math.sin(rot), 0], [-math.sin(rot), math.cos(rot), 0]])
 tmat = np.float32([[1, 0, 0], [0, 1, tr]])
 tfmat = rotmat * tmat
 src_points = np.float32( [[[0, 0]], [[img.shape[1] - 1, 0]], [[0, img.shape[0] - 1]]] )
 dst_points = cv.transform(src=src_points, m=tfmat)

 tform = cv.getAffineTransform(src_points, dst_points)
 fvr = fvr.astype(np.float32)
 img = cv.warpAffine(src=img, M=tform, dsize=(img_w, img_h))
 fvr = cv.warpAffine(src=fvr, M=tform, dsize=(fvr_w, fvr_h), flags=cv.INTER_NEAREST)

 # convert rad to deg
 rot = rot*180/math.pi

 return img, fvr, rot, tr

Answer 1

As Micka commented, last column of the first two rows of rotmat should be the translation.

The translation coefficients of the matrix are 0, 0:

rotmat = np.float32([[math.cos(rot), math.sin(rot), 0], [-math.sin(rot), math.cos(rot), 0]])

That kind of rotation matrix is a "centered transformation" matrix - the transformation is used for transforming (x, y) coordinates, when (0, 0) coordinates is at the center coordinate.

When transforming image coordinated, (0, 0) is the top-left coordinate, so the transformation should not be (0, 0).

---

There are other issues (it hard to follow).

What is the purpose of the following code:

tmat = np.float32([[1, 0, 0], [0, 1, tr]])
tfmat = rotmat * tmat

It looks like transformation chaining, but * in NumPy is element-wise multiplication, and transformation chaining requires matrix multiplication.

tfmat = rotmat @ tmat

It's not going to work, because transformation chaining requires square matrices (3x3).
The last row of an affine transformation matrix is [0, 0, 1].

---

For getting the transformation matrix for rotation, you may use OpenCV method getRotationMatrix2D.
The result applies "top-left" transformation (translation coefficients are not zeros).
The method returns 2x3 matrix - you need to add [0, 0, 1] as the last row, for converting it to transformation matrix.

---

Converting "centered" transformation to "top-left" transformation:

In case you are working with centered transformation, and you need to convert it to "top-left" transformation, the math is not so difficult.
The matrix notation is: trans = cen2top @ centered_trans @ top2cen (when @ applies matrix multiplication).

---

Here is an example for building a "top-left" rotation transformation matrix (please read the comments):

import numpy as np
import cv2

img = cv2.imread('chelsea.png')

rot = 30*np.pi/180

cols, rows = img.shape[1], img.shape[0]

# Centered transformation - translation coefficients are 0
centered_trans = np.float64([[ np.cos(rot), np.sin(rot), 0], 
                             [-np.sin(rot), np.cos(rot), 0],
                             [ 0,           0,           1]])

# Transform top-left coordinate to centered coordinate: x translation = -(cols-1)/2, y translation = -(rows-1)/2
top2cen = np.float64([[1,    0,   -(cols-1)/2],
                      [0,    1,   -(rows-1)/2],
                      [0,    0,    1]])

# Transform centered coordinate to top-left coordinate - x translation = (cols-1)/2, y translation = (rows-1)/2
cen2top = np.float64([[1,      0,    (cols-1)/2],
                      [0,      1,    (rows-1)/2],
                      [0,      0,    1]])

# We need the rotation matrix to be "top left" - coordinate (0, 0) is the top left coordinate:
# Assume column vector u = [x, y, 1] applies top-left coordinates.
# 1. Transform u to centered coordinates:              centerd_u = top2cen*u
# 2. Apply centered_trans on centerd_u:                centerd_v = centered_trans*centerd_u = centered_trans*top2cen*u
# 3. Transform centerd_v to top-left coordinates:      v = cen2top*centerd_v = cen2top*centered_trans*top2cen*u
# The transformation from "top-left" to "top-left" is:                         cen2top*centered_trans*top2cen
trans = cen2top @ centered_trans @ top2cen  # Note: we don't really need to use matrix multiplications for solving this.

# Remove the last row.
rotmat = trans[0:2, :]

# The result should be the same result as cv2.getRotationMatrix2D
# https://docs.opencv.org/master/da/d6e/tutorial_py_geometric_transformations.html
# cols-1 and rows-1 are the coordinate limits.
M = cv2.getRotationMatrix2D(((cols-1)/2.0, (rows-1)/2.0), 30, 1)  # Used as reference


img = cv2.warpAffine(img, rotmat, (cols, rows), cv2.INTER_CUBIC)

cv2.imshow('img', img)
cv2.waitKey()
cv2.destroyAllWindows()

---

Result:

---

I tried to teach you some math...
I hope it helps you solve the issue at the header of your question:

Why is openCV warpAffine setting my entire image to zero?

Warping with wrong transformation matrix may result an image filled with zeros.

---

There might be other issues, but I can't test your code, because you didn't post img, fvr, and edges.