CODEWITHSUNDEEP
python-3.xkerasconv-neural-network
jonathan eslava
jonathan eslava

Reputation: 95

Order of rotated images by using a custom generator

I use a custom image data generator for my project. It receives batches of images and returns [0, 90, 180 and 270] degrees rotated versions of the images with the corresponding class indices {0:0, 1:90, 2:180, 3:270}. Lets assume we have images A, B and C in a batch and images A to Z in the whole data set. All the images are naturally in 0 degree orientation. Initially I returned all the rotated images at the same time. Here is a sample of returned batch: [A0,B0,C0,A1,B1,C1,...,A3,B3,C3]. But this gave me useless results. To compare my approach I trained the same model by using my generator and built in Keras ImageDataGenerator with flow_from_directory. For the built in function I manually rotated original images and stored them in separate folders. Here are the accuracy plots for comparison:Accuracy plots with Keras ImageDataGenerator and custom generator

I used only a few images just to see if there is any difference. From the plots it is obvious that the custom generator is not correct. Hence I think it must return the images as [[A0,B0,C0],[D0,E0,F0]...[...,Z0]], then [[A1,B1,C1],[D1,E1,F1]...[...,Z1]] and so on. To do this I must use the folowing function for multiple times (in my case 4).

    def next(self):
    with self.lock:
        # get input data index and size of the current batch
        index_array = next(self.index_generator)
    # create array to hold the images
    return self._get_batches_of_transformed_samples(index_array)

This function iterates through the directory and returns batches of images. When it reaches to the last image it finishes and the next epoch starts. In my case, in one epoch I want to run this for 4 times by sending the rotation angle as an argument like this: self._get_batches_of_transformed_samples(index_array) , rotation_angle). I was wondering if this is possible or not? If not what could be the solution? Here is the current data generator code:

    def _get_batches_of_transformed_samples(self, index_array):
    # create list to hold the images and labels
    batch_x = []    
    batch_y = []

    # create angle categories corresponding to number of rotation angles
    angle_categories = list(range(0, len(self.target_angles)))

    # generate rotated images and corresponding labels
    for rotation_angle, angle_indice in zip(self.target_angles, angle_categories):
        for i, j in enumerate(index_array):
            if self.filenames is None:
                image = self.images[j]
                if len(image.shape) == 2: image = cv2.cvtColor(image,cv2.COLOR_GRAY2RGB)
            else:
                is_color = int(self.color_mode == 'rgb')
                image = cv2.imread(self.filenames[j], is_color)
                if is_color:
                    if not image is None:
                        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

            # do nothing if the image is none
            if not image is None:
                rotated_im = rotate(image, rotation_angle, self.target_size[:2])
                if self.preprocess_func: rotated_im = self.preprocess_func(rotated_im)

                # add dimension to account for the channels if the image is greyscale
                if rotated_im.ndim == 2: rotated_im = np.expand_dims(rotated_im, axis=2)                                
                batch_x.append(rotated_im)
                batch_y.append(angle_indice)

    # convert lists to numpy arrays
    batch_x = np.asarray(batch_x)
    batch_y = np.asarray(batch_y)        

    batch_y = to_categorical(batch_y, len(self.target_angles))            
    return batch_x, batch_y

def next(self):
    with self.lock:
        # get input data index and size of the current batch
        index_array = next(self.index_generator)
    # create array to hold the images
    return self._get_batches_of_transformed_samples(index_array)

Upvotes: 1

Views: 888

Answers (1)

Kurtis Streutker
Kurtis Streutker

Reputation: 1317

Hmm I would probably do this through keras.utils.Sequence

from keras.utils import Sequence
import numpy as np

class RotationSequence(Sequence):
    def __init__(self, x_set, y_set, batch_size, rotations=(0,90,180,270)):
        self.rotations = rotations
        self.x, self.y = x_set, y_set
        self.batch_size = batch_size

    def __len__(self):
        return int(np.ceil(len(self.x) / float(self.batch_size)))

    def __getitem__(self, idx):
        batch_x = self.x[idx * self.batch_size:(idx + 1) * self.batch_size]
        batch_y = self.y[idx * self.batch_size:(idx + 1) * self.batch_size]

        x, y = [], []
        for rot in self.rotations:
            x += [rotate(cv2.imread(file_name), rotation_angle) for file_name in batch_x]
            y += batch_y

        return np.array(x), np.array(y)

    def on_epoch_end(self):
        shuffle_idx = np.random.permutation(len(self.x))
        self.x, self.y = self.x[shuffle_idx], self.y[shuffle_idx]

And then just pass the batcher to model.fit()

rotation_batcher = RotationSequence(...)
model.fit_generator(rotation_batcher,
    steps_per_epoch=len(rotation_batcher),
    validation_data=validation_batcher,
    epochs=epochs)

This allows you to have more control over the batches being fed into your model. This implementation will almost run. You just need to implement the rotate() function in __getitem__. Also, the batch_size will be 4 times the set size because I just duplicated and rotated each batch. Hope this is helpful to you

Upvotes: 1

Related Questions