ValueError when using keras ImageDataGenerator

Question

I'm trying to run this code from: https://gist.github.com/fchollet/0830affa1f7f19fd47b06d4cf89ed44d

from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K


# dimensions of our images.
img_width, img_height = 150, 150

train_data_dir = './data/train'
validation_data_dir = './data/validation'
nb_train_samples = 2000
nb_validation_samples = 800
epochs = 50
batch_size = 16

if K.image_data_format() == 'channels_first':
    input_shape = (3, img_width, img_height)
else:
    input_shape = (img_width, img_height, 3)

model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size)

model.save_weights('first_try.h5')

I've tried using a few different version of python3, even tried it in a ubuntu virtualbox but I still keep getting this error...

C:\Users\David\Desktop\ai\dcgan_cp.py:70: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators.
  model.fit_generator(
Traceback (most recent call last):
  File "C:\Users\David\Desktop\ai\dcgan_cp.py", line 70, in <module>
    model.fit_generator(
  File "C:\python3\lib\site-packages\keras\engine\training.py", line 2016, in fit_generator
    return self.fit(
  File "C:\python3\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "C:\python3\lib\site-packages\keras_preprocessing\image\iterator.py", line 54, in __getitem__
    raise ValueError('Asked to retrieve element {idx}, '
ValueError: Asked to retrieve element 0, but the Sequence has length 0

I can easily fix the warning by removing _generator but I can't get this code to work. Can anyone help fix this issue? Is there a way to test the directory iterator to see if that is the problem?

Answer 1

This above error ValueError: Asked to retrieve element 0, but the Sequence has length 0 is because there is no images folder defined inside these folders.

train_data_dir = './data/train'
validation_data_dir = './data/validation'

You need to create 2 folders(Cats,Dogs) inside train and 2 folders(Cats,Dogs) inside validation folder
and keep all the training
Cats images inside /data/train/Cats/*.jpg and
Dogs images inside /data/train/Dogs/*.jpg.

and same way all the validation
Cats images inside /data/validation/Cats/*.jpg and
Dogs images inside /data/validation/Dogs/*.jpg.

You can download the dataset from this link.

Make sure your training images are atleast 2000 images and validation image count are atleast 800 images to satisfy this below code for further use:

nb_train_samples = 2000
nb_validation_samples = 800