CODEWITHSUNDEEP
pythonopencvnumpycomputer-vision
Peter Moran
Peter Moran

Reputation: 313

Adding a new channel to an image

I am trying to add a 4th "channel" to an image. Specifically, I have a RGB image and would like append that image matrix with an edge detection layer as found by a Canny filter, which I will then use as an input for a neural network.

I have edge detection working, and I can even append the images, but for some reason, the data 'reverts' after looping. The changes that I make in the size of the image do not stick.

Code

I have three sets of 32x32x3 color images: X_train, X_valid, and X_test. For each one, I am normalizing the image and then appending the gradient. Appending appears to take affect while looping, but the changes do not exist after looping.

Code snippet

import cv2 as cv
example_record = 2

print('X_train is shape {}'.format(X_train.shape))
print('X_valid is shape {}'.format(X_valid.shape))
print('X_test is shape {}'.format(X_test.shape))

# Show before
plt.imshow(X_valid[example_record])
plt.title('Validation Input {} Before Normalization'.format(example_record))

# Normalize
canny_low = 50
canny_high = 100
for dataset in [X_train, X_valid, X_test]:
    for i, img in enumerate(dataset):
        cv.normalize(img, img, 0, 255, cv.NORM_MINMAX)
        edges = cv.Canny(img, canny_low, canny_high)
        edges = np.reshape(edges, (img.shape[0], img.shape[1], 1))
        img = np.concatenate((img, edges),axis=2)
        if i == 0:
            print('img shape after concatenation {}'.format(img.shape))

# Show after
plt.figure()
print('Updated image shape: {}'.format(X_valid[example_record].shape))
plt.imshow(X_valid[example_record])
plt.title('Validation Input {} After Normalization'.format(example_record))

Output

X_train is shape (34799, 32, 32, 3)
X_valid is shape (4410, 32, 32, 3)
X_test is shape (12630, 32, 32, 3)
img shape after concatenation (32, 32, 4)
img shape after concatenation (32, 32, 4)
img shape after concatenation (32, 32, 4)
Updated image shape: (32, 32, 3)

Other Attempts

If I replace img = np.concatenate((img, edges),axis=2) with dataset[i] = np.concatenate((img, edges),axis=2), I get the error:

     21         edges = cv.Canny(img, canny_low, canny_high)
     22         edges = np.reshape(edges, (img.shape[0], img.shape[1], 1))
---> 23         dataset[i] = np.concatenate((img, edges),axis=2)
     24         if i == 0:
     25             print('img shape after concatenation {}'.format(img.shape))

ValueError: could not broadcast input array from shape (32,32,4) into shape (32,32,3)

Upvotes: 2

Views: 1812

Answers (2)

AGN Gazer
AGN Gazer

Reputation: 8378

Ok, my previous answer was not detailed enough for someone's taste and it was down-voted. So, let me provide a "ready" solution:

# Normalize
canny_low = 50
canny_high = 100
X = [X_train, X_valid, X_test]
X_new = [np.empty(x.shape[:-1] + (x.shape[-1] + 1,), dtype=x.dtype) for x in X]

for dataset, dsnew in zip(X, X_new):
    for i, img in enumerate(dataset):
        cv.normalize(img, img, 0, 255, cv.NORM_MINMAX)
        edges = np.expand_dims(cv.Canny(img, canny_low, canny_high), axis=2)
        dsnew[i, :, :, :] = np.concatenate((img, edges), axis=2)

Alternatively, you could expand X_train, X_valid, and X_test before the loop starts and this may save some memory.

Upvotes: 2

AGN Gazer
AGN Gazer

Reputation: 8378

  1. You inner loop

    for i, img in enumerate(dataset):

    is overwriting the value of concatenated img

  2. Your

    print('X_train is shape {}'.format(X_train.shape))
print('X_valid is shape {}'.format(X_train.shape))
print('X_test is shape {}'.format(X_train.shape))

    is printing the value of the shape of X_train.shape!

  3. After the line

    img = np.concatenate(...)

    inside the loop, what do you do with the concatenated image img? Don't you think you have to store the result somehow for the program to "remember" it?

Your second attempt is promising, just do not store new img in the dataset. Define a dataset_new outside the loop (make an empty list or numpy array of the correct shape) and then in the loop do dataset_new[i] = np.concatenate....

Upvotes: 0

Related Questions