Convert Image to Rulinalg Matrix in Rust

Question

I am learning Rust and I want to perform some basic image operations. I am currently reading my images like this: image::open("path/to/img.jpg").unwrap() by using the image crate.

The problem is that I want my image to be in a specific 2d format where there are N=height*weight rows and 3 columns (one for each color). I have found the rulinalg crate but I cannot create a rulinalg::matrix::Matrix from the DynamicImage object. I have tried the following:

// Normalize to [0, 1]
let (width, height) = img.dimensions();
let n = width*height;
let tmp = img.as_rgb8().unwrap().to_vec().iter().map(|&e| e as f32 / 255.0).collect::<Vec<f32>>();
let mat = Matrix::new(n, 3, tmp);

So now I have a matrix of n rows and 3 columns but I am not sure if it is a correct representation. By that, I mean that I am not sure that the first pixel consists of the values mat[[0, 0]], mat[[n, 0]], mat[[n*2, 0]].

So, in order to test it, I thought I should try to recreate the image by using the mat matrix with the following code:

let mut img_buf = image::ImageBuffer::new(width, height);
for i in 0..mat.rows() {
    let color0 = (mat[[i, 0]] * 255.0) as u8;
    let color1 = (mat[[i, 1]] * 255.0) as u8;
    let color2 = (mat[[i, 2]] * 255.0) as u8;
    let x = i as u32 /height;
    let y = i as u32 - x*height;
    let pixel = img_buf.get_pixel_mut(x, y);
    // let image = *pixel;
    *pixel = image::Rgb([color0, color1, color2]);

}
img_buf.save("./tmp.jpg").unwrap();

But the output is only noise (even though the color structure seems to be kept the same). I have tried a lot of things and nothing seems to work. I also tried to find similar "projects" on github but the only relevant thing I found was applying filters to images which simply called functions from the image crate.

Desired Workflow

So, what I want is the following:

1. Read image from path (3d image since it is not grayscale)
2. Convert image to a Matrix (rulinalg::matrix::Matrix)
3. Apply functions such as: element-wise logarith/exponential/square-root, axis-wise sum/max and other similar functions (numpy equivalent: np.log(arr), np.exp(arr), np.sqrt(arr), np.sum(arr, axis=0), np.amax(arr, axis=0))
4. Save the modified matrix as an image.

Reason

My goal is to perform image segmentation and reduce the colors of the image (by clustering).

Question

Does anyone have any idea or pointers of how I can do the above in rust?

Answer 1

Following @Jmb's comment, the reason that the output image was noise was because I have mistaken the axes.

I just want to post what I did in case there is someone else who wants to do something similar. Do keep in mind though that I am new to rust and so I may have made some mistakes.

Using rulinalg

If you want to use the rulinalg crate, the following should work (version 0.4.2):

let img = image::open("path/to/img.jpg").unwrap();
let (width, height) = img.dimensions();

let n = (width * height) as usize;
// Normalize image pixels to [0, 1]
let tmp = img.as_rgb8().unwrap().to_vec().iter().map(|&e| e as f32 / 255.0).collect::<Vec<f32>>();
// Reduce dimensions
let mut mat = Matrix::new(n, 3, tmp);
// Change the array values by using some other method
mat = my_processing_method(mat);
// Image buffer for the new image
let mut img_buf = image::ImageBuffer::new(width, height);
for i in 0..mat.rows() {
    // Move back to the [0, 255] range
    let color0 = (mat[[i, 0]] * 255.0) as u8;
    let color1 = (mat[[i, 1]] * 255.0) as u8;
    let color2 = (mat[[i, 2]] * 255.0) as u8;
    let x = i as u32 % width;
    let y = i as u32 / width;
    let pixel = img_buf.get_pixel_mut(x, y);
    *pixel = image::Rgb([color0, color1, color2]);

}
// Save the updated image
img_buf.save("path/to/new/image.jpg").unwrap();

Using ndarray and ndarray_image:

For ease, you may also use the ndarray-image (version 0.3.0) in order to open the image and save it in an ndarray::Array3 (3d array). If you want to use ndarray instead of the rulinalg crate then you can do the following:

use ndarray::{Array2, Dim};
use ndarray_image::{open_image, save_image, Colors};

let img = open_image("path/to/img.jpg", Colors::Rgb).expect("unable to open input image");
// A vector of 3 spots for each dimension
let sh = img.shape();
let (height, width, colors) = (sh[0] as u32, sh[1] as u32, sh[2]);
let n = (width * height) as usize;
// The dimension of the new 2d array
let new_dim = Dim([n, 3]);
// Normalize to [0, 1] and convert to 1d vector
let img_vec = img.map(|&e| e as f32 / 255_f32).into_raw_vec();
// Convert the 1d vector to the 2d ndarray::Array2
let img_arr = Array2::from_shape_vec(new_dim, img_vec).ok().unwrap();
let mut img_buf = image::ImageBuffer::new(width, height);
for i in 0..img_arr.nrows() {
    let color0 = (img_arr[[i, 0]] * 255.0) as u8;
    let color1 = (img_arr[[i, 1]] * 255.0) as u8;
    let color2 = (img_arr[[i, 2]] * 255.0) as u8;
    let x = i as u32 % width;
    let y = i as u32 / width;
    let pixel = img_buf.get_pixel_mut(x, y);
    *pixel = image::Rgb([color0, color1, color2]);
}
img_buf.save("path/to/new/image.jpg").unwrap();

Hope this is helpful!