CODEWITHSUNDEEP
pythonkeras
Registered User
Registered User

Reputation: 2299

Keras flow_from_dataframe gives 0 images

I am trying to use the flow_from_dataframe method of Keras to read training and testing images.

Both my training and testing images are in same directory, and I read the paths from two different csv files.

My code for reading test images looks like,

# Read test file
testdf = pd.read_csv("test.csv")

# load images
test_datagen = ImageDataGenerator(rescale=1./255)

test_generator = test_datagen.flow_from_dataframe(
    dataframe=testdf, directory=IMAGE_PATH,
    x_col='image_name', y_col=None,
    has_ext=True, target_size=(10,10)
    ,batch_size=32,color_mode='rgb',shuffle=False, class_mode=None)

I get output like this

Found 0 images.

While the similar code for reading training data works properly. I checked if the images exist at the given path, which they do. What are some possible reasons for this error? How can I try to debug the issue?

EDIT: This is a regression task, so all images are in a single directory, and not in subdirectories, as would be expected for a classification task.

EDIT 2: I added usecols=[0] to read_csv, and now test_datagen finds all the images in the directory, and not just the one's that are mentioned in the test.csv file

Upvotes: 2

Views: 2850

Answers (5)

skylon
skylon

Reputation: 1

Okay, so I have been having the same issues. Where my data labels were in a csv file , and the image data in a separate folder.I thought, the issue was being caused by the labels and the images in the folder not aligning properly.Did a whole bunch of stuff to rectify and process the data. It was not the problem. So, anyone who's having issues. I tried @Oussama Ouardini's answer and it worked. Thank you!

I am also going to add - that if you are doing a train and validation split to make sure the initial ImageDataGenerator object you create has the validation split specified.

def extension_train_data(x):
return "xc"+str(x)+".png"


train_df['file_id'] = train_df['file_id'].apply(extension_train_data)

Here is my code -

datagen=ImageDataGenerator(rescale=1./255,validation_split=0.2) 


#rescale all pixel values from 0-255, so after this step all our 
  #pixel values are in range (0,1)
  

train_generator=datagen.flow_from_dataframe(dataframe=train_df,directory='./img_data/', x_col="file_id", y_col="english_cname",
                                        class_mode="categorical",save_to_dir='./new folder/',
                                        target_size=(64,64),subset="training",
                                        seed=42,batch_size=32,shuffle=False)





val_generator=datagen.flow_from_dataframe(dataframe=train_df,directory='./img_d 
               ata/', x_col="file_id", y_col="english_cname",
               class_mode="categorical",
               target_size=(64,64),subset="validation",
               seed=42,batch_size=32,shuffle=False)




print("\n Sanity check Line.--------")

My output was a succesfully validated image files. :)

Found 212 validated image filenames belonging to 88 classes.
Found 52 validated image filenames belonging to 88 classes.

Sanity check Line.----------

I hope someone will find this useful. Cheers!

Upvotes: 0

Oussama Ouardini
Oussama Ouardini

Reputation: 461

I had the same error, What I found is that I missed the directory path, and the image extension that was not in the data frame,

So make sure that your directory path is correct and an extension to your image, as you can do the following:

def extention_train_data(x):
    return x+".jpg"

change the jpg extension if you have an other one.

then you apply this to you data frame:

train_data['image'] = train_data['image_id'].apply(extention_train_data)

once you have the image column containing your image with its extension then

train_generator = datagen.flow_from_dataframe(
train_data,  
directory="/kaggle/input/plant-pathology-2020-fgvc7/images/",
x_col = "image",
y_col = "label",
target_size = size,
class_mode = "binary",
batch_size = batch_size,
subset="training",
shuffle = True,
seed = 42,
)

Upvotes: 0

Lan Dao
Lan Dao

Reputation: 21

I have the same problem. First, make sure you got the absolute path correctly for the parameter directory.

The filename in my df has value image.pgm.png and the actual image file in the folder has the format image.pgm.

  1. I tried to change the filename in df to image.pgm => Still not working
  2. I renamed the image file from image.pgm to image.pgm.png which matches exactly the format in the df => Worked!

Upvotes: 0

lifelong_learner
lifelong_learner

Reputation: 99

I was also facing the same error and found a solution for this. I was using the absolute path, was using correct DataFrame and everything was fine still the code was throwing an error - "image not found".

I inspected and found that my dataframe was containing image names without extension and the images in the folder was having extension also. E.g. The image name in DataFrame was 'abc' but the image in the folder was having a name 'abc.png'. Just add .png in the image names in DataFrame and it will solve your problem. I just tried below code and it worked out..!!!!

def append_ext(fn):
    return fn+".png"
train_valid_data["id_code"]=train_valid_data["id_code"].apply(append_ext)
test_data["id_code"]=test_data["id_code"].apply(append_ext)

Let me know if it solves your problem or if you need any further explanation.

Upvotes: 0

Registered User
Registered User

Reputation: 2299

The issue happens due to NaN's in the dataframe. Ignoring those columns doesn't work. The solution is to replace the NaN's with something else. For example,

testdf = pd.read_csv("test.csv")
testdf.fillna(0, inplace=True)

This replaces the NaN's with 0. Then using ImageDataGenerator as usual works.

Upvotes: 1

Related Questions