CODEWITHSUNDEEP
pythonkeraspickle
a_gdevr
a_gdevr

Reputation: 103

How do you flow_from_directory in keras when your input does not consist in images?

Is there an equivalent to flow_from_directory that works on non-image data? More precisely, I have my data saved as a pickle file consisting of a numpy array. The data is already in the format that I need for the training of the network; is there a way to fit my model by reading each row vector of the pickle file? Or alternatively, saving my row vectors as single pickle files (or other formats) and fitting the model reading them one by one?

Upvotes: 1

Views: 574

Answers (1)

nima farhadi
nima farhadi

Reputation: 698

Yes you can use Generators in keras for this problem.

def Generator(File_address, Batch_Size):
    while True:
      pickle_data = []
      with (open("myfile", "rb")) as openfile: #Read pickle file. this is a sample. you can use your way to read the pickle file.
          while True:
              pickle_data.append(pickle.load(File_address))
      
      for B in range(0, len(pickle_data), Batch_Size):
          X = pickle_data[B:B+Batch_Size]
          Y = Labels[B:B+Batch_Size] #Define your labels
          yield X, Y #Returning data for training.

now you can use your Generator:

train_gen = Generator('Address_to_pickle_file', Batch_Size)

Model.fit(train_gen, epochs=epoch, steps_per_epoch=Number_of_sampels//Batch_Size)

Hope this can be useful. good luck.

Upvotes: 1

Related Questions