CODEWITHSUNDEEP
tensorflowkerasconv-neural-network
Pythonless
Pythonless

Reputation: 53

Iterating over all batches of a generator in keras

I have a dataset formed by images and labels, loaded with a generator such as:

generator = image_generator.flow_from_directory(batch_size=BATCH_SIZE,
                                                           directory=val_dir,
                                                           shuffle=False,
                                                           target_size=(100,100),
                                                           class_mode='categorical')

After prediction with a CNN, I want to iterate through all the results and print the original image and the predicted label.

Using:

x,y = generator.next()

I have managed to do this, but I am limited to the number of elements in a batch of the generator. Trying to print more the loop gets out of index.

How can I iterate through the batches to get all the results, using this method?

Upvotes: 3

Views: 3951

Answers (2)

M.Vu
M.Vu

Reputation: 486

As mentioned in the official link:

for e in range(epochs):
    print('Epoch', e)
    batches = 0
    for x_batch, y_batch in datagen.flow(x_train, y_train, batch_size=32):
        model.fit(x_batch, y_batch)
        batches += 1
        if batches >= len(x_train) / 32:
            # we need to break the loop by hand because
            # the generator loops indefinitely
            break

You can mimic this example code to get each batch in datagen. One import note is that you should set the same seed to keep the order of batch if you need.

Upvotes: 2

Vivek Mehta
Vivek Mehta

Reputation: 2642

you can use itertools.tee which gives you n independent copy of a generator. For example in your case,

generator,gen_copy = itertools.tee(image_generator.flow_from_directory(
                                                  batch_size=BATCH_SIZE,
                                                  directory=val_dir,
                                                  shuffle=False,
                                                  target_size=(100,100),
                                                  class_mode='categorical',seed=0), 
                                                  n=2)

here you will get two copy of same generator. One you can pass to fit_generator and other you can iterate for your purpose. Note that in above code block seed=0 is added to make sure that for each copy shuffle and transformation order is same in case you use them.

Now gen_copy you can use like this:

for x,y in gen_copy:
    print(x,y)

Upvotes: 0

Related Questions