What is the correct way to reshape images after raveling?

Question

At the moment I'm trying to np.ravel() my images so I can use np.append() freely, instead of using np.vstack() which many people here say it's not very fast given the loading/unloading things in memory and I worry it might slow my code down.

My idea was to just flatten the images, append them all and then use np.reshape(appended_images, [512,512,3,-1]) to create the tensor. The tensor is created all right, but upon checkup, the images aren't getting displayed, probably because one of these operations is not working the way I think it should be working.

Checking the final array im_stacked[:,:,:,0] with matplotlib returns a blank image, with a warning of values out of range. Upon inspection of only one channel of the image im_stacked[:,:,0,0] I'm faced with this:

This is just the image repeated over and over. Where is my mistake? Why is there some swapping occurring? Reshaping a single raveled image works fine.

Edit: Minimal code added

import numpy as np
import cv2 as cv
import matplotlib.pyplot as plt

#Image Loading, please use a 512x512x3 image
path = "./path/to/image.png"
im = cv.imread(path)

#Flattening the image
im_raveled = np.ravel(im)

#Starting with an empty array
im_stacked = np.array([])

#For the sake of simplicity this code is just repeated three times
im_stacked = np.append(im_stacked, im_raveled)
im_stacked = np.append(im_stacked, im_raveled)
im_stacked = np.append(im_stacked, im_raveled)

#Using a 515x512x3 image, reshaping the stacked array
im_reshaped = np.reshape(im_stacked, [512,512,3,-1])

#Plotting the images after reshaping
plt.figure()
plt.subplot(1,2,1)

#Plot only the first channel of the first image
plt.imshow(im_reshaped[:,:,0,0])
plt.subplot(1,2,2)

#Plot all channels of the first image
plt.imshow(im_reshaped[:,:,:,0])
plt.show()

Answer 1

Make a sample 3d array:

In [25]: image = np.random.randint(0,256,(512,512,3))

The best way:

In [26]: alist = []
In [27]: for i in range(5):
    ...:     alist.append(image)
    ...: 

It's easy to make an array from such list:

In [28]: np.array(alist).shape
Out[28]: (5, 512, 512, 3)

If you must join them on a new last dimension, use np.stack:

In [29]: np.stack(alist,-1).shape
Out[29]: (512, 512, 3, 5)

np.stack, np.vstack, and even np.append are all covers for np.concatenate. I hate np.append, since it leads too many naive users up the wrong path. It is not an list append clone.

If you must use repeated concatenates do something like:

In [30]: arr = np.zeros((0,512,512,3),image.dtype)

In [31]: arr = np.concatenate([arr,image], axis=0)
Traceback (most recent call last):
  File "<ipython-input-31-1fc945fd1c90>", line 1, in <module>
    arr = np.concatenate([arr,image], axis=0)
  File "<__array_function__ internals>", line 5, in concatenate
ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 4 dimension(s) and the array at index 1 has 3 dimension(s)

oops, even with experience I have troubles getting that started.

In [32]: arr = np.concatenate([arr,image[None,...]], axis=0)
In [33]: arr.shape
Out[33]: (1, 512, 512, 3)
In [34]: arr = np.concatenate([arr,image[None,...]], axis=0)
In [35]: arr.shape
Out[35]: (2, 512, 512, 3)

Repeated concatenate is slow. concatenate takes a whole list of arrays, and should be used as such. Don't try to replicate list code in lists!

List append is easy because there's an obvious "empty" list, and you can efficiently add references to it. Arrays don't have an equivalent "empty" array. Dimensions matter, right from the start. I had to start with a (0,512,512,3) shape. If you don't know the needed dimensions, then don't take this approach.

---

As for your title question, this might work:

im_reshaped = np.reshape(im_stacked, [-1,512,512,3])

With the repeated np.append, you joined the ravelled arrays end to end, [(786432,),(786432,),(786432,),...]. Effectively the new dimension is a leading one, not a trailing one. It's a crude way of performing the list append and array build that I started with.