How can i speed up operations with NumPy array in Python 2.7

Question

I try to process many images which represented as NumPy array, but it takes too long. that's what im trying to do

# image is a list with images
max = np.amax(image[k])# k is current image index in loop
# here i try to normalize SHORT color to BYTE color and make it fill all range from 0 to 255
# in images max color value is like 30000 min is usually 0
i = 0
while i < len(image[k]):
    j = 0
    while j < len(image[k][i]):
        image[k][i][j] = float(image[k][i][j]) / (max) * 255
        j += 1
    i += 1

if i only read images (170 in total (images is 512x512)) without it takes about 7 secs, if i do this normalization it takes 20 mins. And it's all over in code. Here i try to make my image colored

maskLoot1=np.zeros([len(mask1), 3*len(mask1[0])])
for i in range(len(mask1)):
    for j in range(len(mask1[0])):
        maskLoot1[i][j*3]=mask1[i][j]
        maskLoot1[i][j*3+1]=mask1[i][j]
        maskLoot1[i][j*3+2]=mask1[i][j]

Next i try to replace selected region pixels with colored ones, for example 120 (grey) -> (255 40 0) in rgb model.

            for i in range(len(mask1)):
                for j in range(len(mask1[0])):
                #mask is NumPy array with selected pixel painted in white (255)
                    if (mask[i][j] > 250):
                        maskLoot1[i][j * 3] = lootScheme[mask1[i][j]][1] #red chanel
                        maskLoot1[i][j * 3+1] = lootScheme[mask1[i][j]][2] #green chanel
                        maskLoot1[i][j * 3+2] = lootScheme[mask1[i][j]][3] #bluechanel

And it also takes much time, not 20 min but long enouch to make my script lag. consider it's just 2 of many my operations on arrays, and if for second case we can use some bultin function for others is very unlikely. So is there a way to speed up my sode?

Answer 1

First thing, consider moving to Python 3.*. Numpy is dropping support for Python Numpy is dropping support for Python 2.7 from 2020.

For your code questions. You are missing the point of using Numpy below. Numpy is compiled from lower level libraries and it runs very fast, you should not loop over indices in Python, you should throw matrices to Numpy.

Question 1 Normalization is very fast using a listcomp and an np.array

import numpy as np
import time

# create dummy image structure (k, i, j, c) or (k, i, j)
# k is image index, i is row, j is columns, c is channel RGB
images = np.random.uniform(0, 30000, size=(170, 512, 512))

t_start = time.time()
norm_images = np.array([(255*images[k, :, :]/images[k, :, :].max()).astype(int) for k in range(170)])
t_end = time.time()

print("Processing time = {} seconds".format(t_end-t_start))
print("Input shape = {}".format(images.shape))
print("Output shape = {}".format(norm_images.shape))
print("Maximum input value = {}".format(images.max()))
print("Maximum output value = {}".format(norm_images.max()))

That creates the following output

Processing time = 0.2568979263305664 seconds
Input shape = (170, 512, 512)
Output shape = (170, 512, 512)
Maximum input value = 29999.999956185838
Maximum output value = 255

It takes 0.25 seconds!

Question 2 Not sure what you meant here but if you want to clone the values of a monochromatic image to RGB values you can do it like this

# coloring (by copying value and keeping your structure)
color_img = np.array([np.tile(images[k], 3) for k in range(170)])
print("Output shape = {}".format(color_img.shape))

Which produces

Output shape = (170, 512, 1536)

If you instead would like to keep a (c, i, j, k) structure

color_img = np.array([[images[k]]*3 for k in range(170)])  # that creates (170, 3, 512, 512)
color_img = np.swapaxes(np.swapaxes(color_img, 1,2), 2, 3) # that creates (170, 512, 512, 3)

All this takes 0.26 seconds!

Question 3 Coloring certain regions, I would use a function again and a listcomp. Since this is an example I have used a default colouring of (255, 40, 0) but you can use anything, including a LUT.

# create mask of zeros and ones
mask = np.floor(np.random.uniform(0,256, size=(512,512)))
default_scheme = (255, 40, 0)

def substitute(cimg, mask, scheme):
  ind = mask > 250
  cimg[ind, :] = scheme
  return cimg

new_cimg = np.array([substitute(color_img[k], mask, default_scheme) for k in range(170)])

Answer 2

For your mask-making code try this replacement to loops:

maskLoot1 = np.dstack(3*[mask1]).reshape((mask1.shape[0],3*mask1.shape[1]))

There are many other ways/variations of achieving the above, e.g.,

maskLoot1 = np.tile(mask1[:,:,None], 3).reshape((mask1.shape[0],3*mask1.shape[1]))

As for the first part of your question the best answer is in the first comment to your question by @furas

Answer 3

In general for-loops are significantly faster than while-loops. Also using a function for

maskLoot1[i][j*3]=mask1[i][j]
maskLoot1[i][j*3+1]=mask1[i][j]
maskLoot1[i][j*3+2]=mask1[i][j]

and calling the function in the loop should speed up the process significantly.