CODEWITHSUNDEEP
pythontensorflowkeras
AaronDT
AaronDT

Reputation: 4060

Keras - Load images from generator into memory

I would like to load my images into memory utilizing the image augmentation opions provided by the keras ImageDataGenerator. As such, I am creating my generator like so:

testgen = ImageDataGenerator(preprocessing_function=keras.applications.mobilenet.preprocess_input)

test_generator = testgen.flow_from_dataframe(dataframe=df_test, 
                                             classes=class_labels, 
                                             directory=data_dir,
                                             x_col=training_on, 
                                             y_col=target, 
                                             has_ext=True, 
                                             class_mode="categorical", 
                                             target_size=(224,224), 
                                             batch_size=batch_size,
                                             seed = 1,
                                             shuffle=False)

Now I can get a single batch using:

x,y = next(test_generator)

However, I would like to store the entire dataset (possibly augmented) into x and y. How can I achieve this?

Upvotes: 0

Views: 502

Answers (1)

Daniel Möller
Daniel Möller

Reputation: 86600

xTrain = list()
yTrain = list()
for i in range(len(test_generator)):
    x,y = test_generator[i] 
    xTrain.append(x)
    yTrain.append(y)

xTrain = np.array(xTrain)
yTrain = np.array(yTrain)

Alternative:

xTrain = list()
yTrain = list()
for i in range(number_of_batches):
    x,y = next(test_generator)
    xTrain.append(x)
    yTrain.append(y)

xTrain = np.array(xTrain)
yTrain = np.array(yTrain)

Notice that this will not result in significant augmentation. You will end up with the same number of samples as the original data.

For augmentation to really work you need to train over and over with this generator so it produces many different random versions of the same images.

Upvotes: 1

Related Questions