CODEWITHSUNDEEP
pythonkeras
Joshhh
Joshhh

Reputation: 465

Using Keras callbacks to alter the dataset

I want to change parts of my dataset at every epoch. As written in the Keras documentation, in order to create a callback, I need to create a class. So I started by writing

class AlterDataset(keras.callbacks.Callback):
    def on_epoch_end(self,epoch,logs={}):
        #???

but then I realized I have no way to access the dataset of the model. Can this be done using callbacks?

I have also seen this entry, but I didn't quite understand this. I have a model architecture already in place, and I use Model, not Sequential.

Upvotes: 3

Views: 1035

Answers (1)

markuscosinus
markuscosinus

Reputation: 2267

You can implement a Sequence that loads the data for your model during training. It has an on_epoch_end method in which you could change your data before the next epoch is started.

Rough example:

class MySequence(Sequence):

    def __init__(self, batchSize): # you can add parameters here
        self.batchSize = batchSize
        self.xTrain = loadxData() # load your x data here
        self.yTrain = loadyData() # load your y data here

    def __len__(self):
        return self.xData.shape[0]//self.batchSize

    def __getitem__(self, index):
        return self.xTrain[index*self.batchSize:(index+1)*self.batchSize:]

    def on_epoch_end(self):
        self.xTrain, self.yTrain = changeData(self.xTrain, self.yTrain) # change your data here

Then you can fit your model using fit_generator.

Upvotes: 1

Related Questions