Error: expected conv3d_1_input to have 5 dimensions, but got array with shape (10, 224, 224, 3)

Question

I'm trying to train a Neural Network on a dataset for liveness anti-spoofing. I have some videos in two folders named genuine and fake. I have extracted 10 frames of each video and saved them in two folders with aforementioned names under a new directory tarining.

--/training/
----/genuine/   #containes 10frame*300videos=3000images
----/fake/   #containes 10frame*800videos=8000images

I designed the following 3D Convent using Keras as my first try, but after running it, it throws the following exception:

from keras.preprocessing.image import ImageDataGenerator
from keras import Model, optimizers, activations, losses, regularizers, backend, Sequential
from keras.layers import Dense, MaxPooling3D, AveragePooling3D, Conv3D, Input, Flatten, BatchNormalization

BATCH_SIZE = 10
TARGET_SIZE = (224, 224)

train_datagen = ImageDataGenerator(rescale=1.0/255,
                                   data_format='channels_last',
                                   validation_split=0.2,
                                   shear_range=0.0,
                                   zoom_range=0,
                                   horizontal_flip=False,
                                   featurewise_center=False,
                                   featurewise_std_normalization=False,
                                   width_shift_range=False,
                                   height_shift_range=False)

train_generator = train_datagen.flow_from_directory("./training/",
                                                    target_size=TARGET_SIZE,
                                                    batch_size=BATCH_SIZE,
                                                    class_mode='binary',
                                                    shuffle=False,
                                                    subset='training')

validation_generator = train_datagen.flow_from_directory("./training/",
                                                    target_size=TARGET_SIZE,
                                                    batch_size=BATCH_SIZE,
                                                    class_mode='binary',
                                                    shuffle=False,
                                                    subset='validation')

SHAPE = (10, 224, 224, 3)
model = Sequential()
model.add(Conv3D(filters=128, kernel_size=(1, 3, 3), data_format='channels_last', activation='relu', input_shape=(10, 224, 224, 3)))
model.add(MaxPooling3D(data_format='channels_last', pool_size=(1, 2, 2)))
model.add(Conv3D(filters=64, kernel_size=(2, 3, 3), activation='relu'))
model.add(MaxPooling3D(pool_size=(1, 2, 2)))
model.add(Conv3D(filters=32, kernel_size=(2, 3, 3), activation='relu'))
model.add(Conv3D(filters=32, kernel_size=(2, 3, 3), activation='relu'))
model.add(MaxPooling3D(pool_size=(1, 2, 2)))
model.add(Conv3D(filters=16, kernel_size=(2, 3, 3), activation='relu'))
model.add(Conv3D(filters=16, kernel_size=(2, 3, 3), activation='relu'))
model.add(AveragePooling3D())
model.add(BatchNormalization())
model.add(Dense(32, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.summary()
model.compile(optimizer=optimizers.adam(), loss=losses.binary_crossentropy, metrics=['accuracy'])
model.fit_generator(train_generator, steps_per_epoch=train_generator.samples/train_generator.batch_size, epochs=5, validation_data=validation_generator, validation_steps=validation_generator.samples/validation_generator.batch_size)
model.save('3d.h5')

Here is the Error:

ValueError: Error when checking input: expected conv3d_1_input to have 5 dimensions, but got array with shape (10, 224, 224, 3)

And this is the output of model.summary()

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv3d_1 (Conv3D)            (None, 10, 222, 222, 128) 3584      
_________________________________________________________________
max_pooling3d_1 (MaxPooling3 (None, 10, 111, 111, 128) 0         
_________________________________________________________________
conv3d_2 (Conv3D)            (None, 9, 109, 109, 64)   147520    
_________________________________________________________________
max_pooling3d_2 (MaxPooling3 (None, 9, 54, 54, 64)     0         
_________________________________________________________________
conv3d_3 (Conv3D)            (None, 8, 52, 52, 32)     36896     
_________________________________________________________________
conv3d_4 (Conv3D)            (None, 7, 50, 50, 32)     18464     
_________________________________________________________________
max_pooling3d_3 (MaxPooling3 (None, 7, 25, 25, 32)     0         
_________________________________________________________________
conv3d_5 (Conv3D)            (None, 6, 23, 23, 16)     9232      
_________________________________________________________________
conv3d_6 (Conv3D)            (None, 5, 21, 21, 16)     4624      
_________________________________________________________________
average_pooling3d_1 (Average (None, 2, 10, 10, 16)     0         
_________________________________________________________________
batch_normalization_1 (Batch (None, 2, 10, 10, 16)     64        
_________________________________________________________________
dense_1 (Dense)              (None, 2, 10, 10, 32)     544       
_________________________________________________________________
dense_2 (Dense)              (None, 2, 10, 10, 1)      33        
=================================================================
Total params: 220,961
Trainable params: 220,929
Non-trainable params: 32
__________________________________________________________

I'd appreciate any help to fix the exception. By the way, I'm using TensorFlow as backend if it helps to solve the problem.

Answer 1

As @thushv89 mentioned in the comments Keras has no build-in video generator which causes a lot of problems for those who will work with big video datasets. Therefore, I wrote a simple VideoDataGenerator which works almost as simple as ImageDataGenerator. The script could be found here on my github in case someone needs it in the future.