CODEWITHSUNDEEP
pythonnumpymachine-learningdeep-learningkeras
Robert Valencia
Robert Valencia

Reputation: 1752

Keras Image Preprocessing: Tuple Index Out of Range

The goal of the script is to use Keras' existing image preprocessing module for video data augmentation. In this prototype, a sample video is split into an array of frames and processed, where the final steps involve performing random rotations, shifts, shears, and zooms:

from keras import backend as K
from keras.preprocessing.image import random_rotation, random_shift, random_shear, random_zoom
K.set_image_dim_ordering("th")

import cv2
import numpy as np

video_file_path = "./training-data/yes/1.mov"
samples_generated_per_sample = 10
self_rows = 100
self_columns = 150
self_frames_per_sequence = 45

# haar cascades for localizing oral region
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
mouth_cascade = cv2.CascadeClassifier('haarcascade_mcs_mouth.xml')

video = cv2.VideoCapture(video_file_path)
success, frame = video.read()

frames = []
success = True

# convert to grayscale, localize oral region, equalize dimensions, 
# normalize pixels, equalize lengths, and accumulate valid frames 
while success:
  success, frame = video.read()
  if success:
    # convert to grayscale
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # localize single facial region
    faces_coords = face_cascade.detectMultiScale(frame, 1.3, 5)
    if len(faces_coords) == 1:
      face_x, face_y, face_w, face_h = faces_coords[0]
      frame = frame[face_y:face_y + face_h, face_x:face_x + face_w]

      # localize oral region
      mouth_coords = mouth_cascade.detectMultiScale(frame, 1.3, 5)
      threshold = 0
      for (mouth_x, mouth_y, mouth_w, mouth_h) in mouth_coords:
        if (mouth_y > threshold):
            threshold = mouth_y
            valid_mouth_coords = (mouth_x, mouth_y, mouth_w, mouth_h)
        else:
            pass
      mouth_x, mouth_y, mouth_w, mouth_h = valid_mouth_coords
      frame = frame[mouth_y:mouth_y + mouth_h, mouth_x:mouth_x + mouth_w]

      frames.append(frame)

    # ignore multiple facial region detections
    else:
        pass

# pre-pad short sequences and equalize sequence lengths
if len(frames) < self_frames_per_sequence:
    frames = [frames[0]]*(self_frames_per_sequence - len(frames)) + frames
frames = frames[0:self_frames_per_sequence]
frames = np.asarray(frames)

rotated_frames = random_rotation(frames, rg=45)
shifted_frames = random_shift(rotated_frames, wrg=0.25, hrg=0.25)
sheared_frames = random_shear(shifted_frames, intensity=0.79)
zoomed_frames = random_zoom(sheared_frames, zoom_range=(1.25, 1.25))

When the script is the run, the following error appears: Output Error

Upvotes: 0

Views: 1872

Answers (2)

Robert Valencia
Robert Valencia

Reputation: 1752

The problem was due to the unequal frame dimensions. The solution was to equalize the frame dimensions first before applying transformations:

# pre-pad short sequences, equalize frame dimensions, and equalize sequence lengths
if len(frames) < self_frames_per_sequence:
    frames = [frames[0]]*(self_frames_per_sequence - len(frames)) + frames
frames = frames[0:self_frames_per_sequence]
frames = [cv2.resize(frame, (self_columns, self_rows)).astype('float32') for frame in frames]
frames = np.asarray(frames)

rotated_frames = random_rotation(frames, rg=45)
shifted_frames = random_shift(rotated_frames, wrg=0.25, hrg=0.25)
sheared_frames = random_shear(shifted_frames, intensity=0.79)
zoomed_frames = random_zoom(sheared_frames, zoom_range=(1.25, 1.25))

Upvotes: 0

Benjamin
Benjamin

Reputation: 11860

Look at the parameters: https://github.com/fchollet/keras/blob/master/keras/preprocessing/image.py#L25.

You are giving frames as a flat array, but it is expecting an array with at least three axes, so it can take row_axis=1, col_axis=2, by default. Either specify those parameters correctly, or provide with a correctly shaped array.

Upvotes: 2

Related Questions