Keras ImageGenerator : IndexError: list index out of range

Question

I loaded the data from kaggle kernel to my machine for reprodicing, now the code is not working, but works on the keras on same python environment.

Here is the code and the bug.

def flow_from_dataframe(img_data_gen, in_df, path_col, y_col, **dflow_args):
    base_dir = os.path.dirname(in_df[[path_col]].values[0])
    print('## Ignore next message from keras, values are replaced anyways')
    df_gen = img_data_gen.flow_from_directory(base_dir, 
                                     class_mode = 'sparse',
                                    **dflow_args)
    df_gen.filenames = in_df[path_col].values
    df_gen.classes = np.stack(in_df[y_col].values)
    df_gen.samples = in_df.shape[0]
    df_gen.n = in_df.shape[0]
    df_gen._set_index_array()
    df_gen.directory = '' # since we have the full path
    print('Reinserting dataframe: {} images'.format(in_df.shape[0]))
    return df_gen


train_gen = flow_from_dataframe(core_idg, train_df, 
                             path_col = 'path',
                            y_col = 'disease_vec', 
                            target_size = IMG_SIZE,
                             color_mode = 'rgb',
                            batch_size = 32)

valid_gen = flow_from_dataframe(core_idg, valid_df, 
                             path_col = 'path',
                            y_col = 'disease_vec', 
                            target_size = IMG_SIZE,
                             color_mode = 'rgb',
                            batch_size = 256) # we can use much larger batches for evaluation
# used a fixed dataset for evaluating the algorithm
test_X, test_Y = next(flow_from_dataframe(core_idg, 
                               valid_df, 
                             path_col = 'path',
                            y_col = 'disease_vec', 
                            target_size = IMG_SIZE,
                             color_mode = 'rgb',
                            batch_size = 1024)) # one big batch

t_x, t_y = next(train_gen)
fig, m_axs = plt.subplots(4, 4, figsize = (16, 16))

I am able to see the paths and path lists, but not sure, where it is coming from.

Found 0 images belonging to 0 classes.
Reinserting dataframe: 10000 images
Traceback (most recent call last):
  File "main.py", line 181, in <module>
    batch_size = 32)) # one big batch
  File "/home/user/.conda/envs/test_env/lib/python3.6/site-packages/keras_prepro         cessing/image/iterator.py", line 104, in __next__
    return self.next(*args, **kwargs)
  File "/home/user/.conda/envs/test_env/lib/python3.6/site-packages/keras_prepro         cessing/image/iterator.py", line 116, in next
    return self._get_batches_of_transformed_samples(index_array)
  File "/home/user/.conda/envs/test_env/lib/python3.6/site-packages/keras_prepro         cessing/image/iterator.py", line 227, in _get_batches_of_transformed_samples
    img = load_img(filepaths[j],

Answer 1

The error occurs because filepaths is empty list. Add to filepaths the images paths:

df_gen.filepaths.extend(df_gen.filenames)