CODEWITHSUNDEEP
pythontensorflowkeras
AlefSin
AlefSin

Reputation: 1095

Keras model.fit() with tf.dataset fails while using tf.train works fine

Summary: according to the documentation, Keras model.fit() should accept tf.dataset as input (I am using TF version 1.12.0). I can train my model if I manually do the training steps but using model.fit() on the same model, I get an error I cannot resolve.

Here is a sketch of what I did: my dataset, which is too big to fit in the memory, consists of many files each with different number of rows of (100 features, label). I'd like to use tf.data to build my data pipeline:

def data_loader(filename):
    '''load a single data file with many rows'''
    features, labels = load_hdf5(filename)
    ...
    return features, labels

def make_dataset(filenames, batch_size):
    '''read files one by one, pick individual rows, batch them and repeat'''
    dataset = tf.data.Dataset.from_tensor_slices(filenames)
    dataset = dataset.map(      # Problem here! See edit for solution
        lambda filename: tuple(tf.py_func(data_loader, [filename], [float32, tf.float32])))
    dataset = dataset.flat_map(
        lambda features, labels: tf.data.Dataset.from_tensor_slices((features, labels)))
    dataset = dataset.batch(batch_size)
    dataset = dataset.repeat()
    dataset = dataset.prefetch(1000)
    return dataset

_BATCH_SIZE = 128
training_set = make_dataset(training_files, batch_size=_BATCH_SIZE)

I'd like to try a very basic logistic regression model:

inputs = tf.keras.layers.Input(shape=(100,))
outputs = tf.keras.layers.Dense(1, activation='softmax')(inputs)
model = tf.keras.Model(inputs, outputs)

If I train it manually everything works fine, e.g.:

labels = tf.placeholder(tf.float32)
loss = tf.reduce_mean(tf.keras.backend.categorical_crossentropy(labels, outputs))
train_step = tf.train.GradientDescentOptimizer(.05).minimize(loss)

iterator = training_set.make_one_shot_iterator()
next_element = iterator.get_next()
init_op = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init_op)
    for i in range(training_size // _BATCH_SIZE):
        x, y = sess.run(next_element)
        train_step.run(feed_dict={inputs: x, labels: y})

However, if I instead try to use model.fit like this:

model.compile('adam', 'categorical_crossentropy', metrics=['acc'])
model.fit(training_set.make_one_shot_iterator(),
          steps_per_epoch=training_size // _BATCH_SIZE,
          epochs=1,
          verbose=1)

I get an error message ValueError: Cannot take the length of Shape with unknown rank. inside the keras'es _standardize_user_data function.

I have tried quite a few things but could not resolve the issue. Any ideas?

Edit: based on @kvish's answer, the solution was to change the map from a lambda to a function that would specify the correct tensor dimensions, e.g.:

def data_loader(filename):
    def loader_impl(filename):
        features, labels, _ = load_hdf5(filename)
        ...
        return features, labels

    features, labels = tf.py_func(loader_impl, [filename], [tf.float32, tf.float32])
    features.set_shape((None, 100))
    labels.set_shape((None, 1))
    return features, labels

and now, all needed to do is to call this function from map:

dataset = dataset.map(data_loader)

Upvotes: 2

Views: 1591

Answers (1)

kvish
kvish

Reputation: 1012

Probably tf.py_func produces an unknown shape which Keras cannot infer. We can set the shape of the tensor returned by it using set_shape(your_shape) method and that would help Keras infer the shape of the result.

Upvotes: 3

Related Questions