Rotating a reshaped image as a matrix operation

Question

I have a gray scale image that I want to rotate. However, I need to do optimization on it. Therefore, I cannot use pillow or opencv. I want to reshape this image using python with numpy.reshape into an one dimensional vector (where I use the default settings C-style reshape). And thereafter, I want to rotate this image around a point using matrix multiplication and addition, i.e. it should be something like

rotated_image_vector = A @ vector + b # (or the equivalent in homogenious coordinates).

After this operation I want to reshape the outcome back to two dimensions and have the rotated image. It would be best if it would as well use linear interpolation between the pixels that do not fit exactly to an other pixel.

The mathematical theory tells it is possible, and I believe there is a very elegant solution to this problem, but I do not see how to create this matrix. Did anyone already have this problem or sees an immediate solution?

Thanks a lot, Eike

Answer 1

Thank you for your answer!

But actually it is not a misconception that you could let this roation be represented by a matrix multiplication with the reshaped vector. I used your code to generate such a matrix (its surely not the most efficient way but it works, most likely you see a more efficient implementation immediately XD. You see I really need it as a matix multiplication :-D). What I basically did is to generate the representation matrix of the linear transformation, by computing how every of the 100*100 basis images (i.e. the image with zeros everywhere und a one) is mapped by your transformation.

import sys
import numpy as np
import matplotlib.pyplot as plt
import itertools

angle = 2*np.pi/6
image_expl = plt.imread('wikipedia.jpg')
image_expl = image_expl[:,:,0]
plt.imshow(image_expl)
plt.title("Image")
plt.show()
image_shape = image_expl.shape
pixel_number = image_shape[0]*image_shape[1]
rot_mat = np.zeros((pixel_number,pixel_number))
for i in range(pixel_number):
    vector = np.zeros(pixel_number)
    vector[i] = 1
    image = vector.reshape(*image_shape)
    fixed_point = np.array(image.shape, dtype='float')/2
    points = np.moveaxis(np.indices(image.shape),0,-1).reshape(-1,2)
    a = -angle
    A = np.array([[np.cos(a),-np.sin(a)],[np.sin(a),np.cos(a)]])
    rotated_coordinates = (A@(points-fixed_point.reshape(1,2)).T).T+fixed_point.reshape(1,2)
    x,y = rotated_coordinates.T
    image = image.astype('float')
    weights_x = [(1-(x % 1)).reshape(*x.shape), (x % 1).reshape(*x.shape)]
    weights_y = [(1-(y % 1)).reshape(*x.shape), (y % 1).reshape(*x.shape)]
    start_x = np.floor(x)
    start_y = np.floor(y)
    transformed_image_returned = sum([image[np.clip(np.floor(start_x + x), 0, image.shape[0]-1).astype('int'),
                         np.clip(np.floor(start_y + y), 0, image.shape[1]-1).astype('int')] * weights_x[x]*weights_y[y] 
                   for x,y in itertools.product(range(2),range(2))])
    rot_mat[:,i] = transformed_image_returned
    if i%100 == 0: print(int(100*i/pixel_number), "% finisched")


plt.imshow((rot_mat @ image_expl.reshape(-1)).reshape(image_shape))

Thank you again :-)

Answer 2

I like your approach but there is a slight misconception in it. What you want to transform are not the pixel values themselves but the coordinates. So you don't reshape your image but rather do a np.indices on it to obtain coordinates to each pixel. For those a rotation around a point looks like

rotation_matrix@(coordinates-fixed_point)+fixed_point

except that I have to transpose a bit to get the dimensions to align. The cove below is a slight adoption of my code in this answer.

As an example I am going to use the Wikipedia-logo-v2 by Nohat. It is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.

First I read in the picture, swap x and y axis to not get mad and rotate the coordinates as described above.

import numpy as np
import matplotlib.pyplot as plt
import itertools

image = plt.imread('wikipedia.jpg')
image = np.swapaxes(image,0,1)/255

fixed_point = np.array(image.shape[:2], dtype='float')/2
points = np.moveaxis(np.indices(image.shape[:2]),0,-1).reshape(-1,2)
a = 2*np.pi/8
A = np.array([[np.cos(a),-np.sin(a)],[np.sin(a),np.cos(a)]])
rotated_coordinates = (A@(points-fixed_point.reshape(1,2)).T).T+fixed_point.reshape(1,2)

Now I set up a little class to interpolate between the pixels that do not fit exactly to an other pixel. And finally I swap the axis back and plot it.

class Image_knn():
   def fit(self, image):
       self.image = image.astype('float')

   def predict(self, x, y):
       image = self.image
       weights_x = [(1-(x % 1)).reshape(*x.shape,1), (x % 1).reshape(*x.shape,1)]
       weights_y = [(1-(y % 1)).reshape(*x.shape,1), (y % 1).reshape(*x.shape,1)]
       start_x = np.floor(x)
       start_y = np.floor(y)
       return sum([image[np.clip(np.floor(start_x + x), 0, image.shape[0]-1).astype('int'),
                         np.clip(np.floor(start_y + y), 0, image.shape[1]-1).astype('int')] * weights_x[x]*weights_y[y] 
                   for x,y in itertools.product(range(2),range(2))])

   
image_model = Image_knn()
image_model.fit(image)

transformed_image = image_model.predict(*rotated_coordinates.T).reshape(*image.shape)
plt.imshow(np.swapaxes(transformed_image,0,1))

And I get a result like this

Possible Issue

The artifact in the bottom left that looks like one needs to clean the screen comes from the following problem: When we rotate it can happen that we don't have enough pixels to paint the lower left. What we do by default in image_knn is to clip the coordinates to an area where we have information. That means when we ask image knn for pixels coming from outside the image it gives us the pixels at the boundary of the image. This looks good if there is a background but if an object touches the edge of the picture it looks odd like here. Just something to keep in mind when using this.