CODEWITHSUNDEEP
pythontensorflowkeraserror-handlinggenerator
ubiqum
ubiqum

Reputation: 115

Keras. Failed to find data adapter. Custom generator

I'm trying to fit my keras model with a custom generator. The error:

Failed to find data adapter that can handle input: <class 'custom_generator.MyCustomGenerator'>, <class 'NoneType'>

Code i'm trying to use (tried both methods fit and fit_generator)

my_gen = MyCustomGenerator(path_to_dataset, batch_size=10)
model.fit(my_gen, verbose=1, epochs=10, validation_freq=0.1, callbacks=tensorboard_callback)

This is my class for custom generator.

I've tried import tensorflow.keras.utils.Sequence

and

tensorflow.python.keras.utils.data_utils.Sequence - still no result

import math
import tensorflow as tf
import numpy as np
import glob
from process_image import process_images
import keras


class MyCustomGenerator(tf.keras.utils.Sequence):

    def __init__(self, path_to_data, batch_size):
        self.path_to_data = path_to_data
        self.batch_size = batch_size
        self.files = []
        for file in glob.glob(f'{path_to_data}/*.npy'):
            self.files.append(file)

    def __len__(self):
        return math.ceil(len(self.files) / self.batch_size)

    def __getitem__(self, idx):
        if idx * self.batch_size % 500 == 0:
            data = np.load(self.files[int(idx // (500 / self.batch_size))], allow_pickle=True)
            self.data = data

        batch_x = self.data[idx * self.batch_size: (idx + 1) * self.batch_size, 0]
        batch_y = self.data[idx * self.batch_size: (idx + 1) * self.batch_size, 1]
        #batch_x = process_images(batch_x)
        return np.array(batch_x), np.array(batch_y)

keras 2.4.3

tensorflow 2.5.0rc1

Found very similar problem in this thread

Keras fit generator - ValueError: Failed to find data adapter that can handle input

But solution is not working for me

I would be very grateful for any help

Upvotes: 0

Views: 409

Answers (1)

ubiqum
ubiqum

Reputation: 115

@Dr.Snoopy Thank you, i did pip uninstall keras and replaced

from keras.layers

to

from tensorflow.keras.layers

in other modules. Works fine now

Upvotes: 1

Related Questions