CODEWITHSUNDEEP
pythontensorflowkerasconv-neural-networkgenerator
Tobitor
Tobitor

Reputation: 1508

ImageDataGenerator - Preprocessing X_train

I have two datasets. The first one contains the image data paths, so the path to my inputs X_train. The second dataset contains the labels, they are one hot encoded and in a special format, their shape is 3-dimensional (number of images, length of the label, character possibilities), i.e. (n, 8, 36) for my dataset. The labels are the y_train data.

The shape of the labels is the reason why I am looking for why I am looking for a method to read in and preprocess X_train batchwise and separately from the y_train data. Is there such a method or do you know how to work around this problem?

Thanks a lot!

Upvotes: 0

Views: 139

Answers (1)

SajanGohil
SajanGohil

Reputation: 969

You can use a custom keras generator by creating a class which inherits from Sequence class

Another [SO answer with more details[(Clarification about keras.utils.Sequence)

Here's an example

class Custom_Generator(keras.utils.Sequence) :
    def __init__(self,...,datapath, batch_size, ..) :

    def __len__(self) :
        #calculate data len, something like len(train_labels)


    def load_and_preprocess_function(self, label_names, ...):
        #do something...
        #load data for the batch using label names with whatever library

    def __getitem__(self, idx) :
        batch_y = train_labels[idx:idx+batch_size]
        batch_x = self.load_and_preprocess_function()
        return ( batch_x, batch_y )

Upvotes: 1

Related Questions