CODEWITHSUNDEEP
tensorflowkerasdeep-learningtensorflow2.0tensorflow-datasets
Mehdi
Mehdi

Reputation: 1296

Passing x_train as a list of numpy arrays to tf.data.Dataset is not working

My problem is that x_train in tf.data.Dataset.from_tensor_slices(x_train, y_train) needs to be a list. When I use the following lines to pass [x1_train,x2_train] to tensorflow.data.Dataset.from_tensor_slices, then I get error (x1_train, x2_train and y_train are numpy arrays):

Train=tensorflow.data.Dataset.from_tensor_slices(([x1_train,x2_train], y_train)).batch(batch_size)

Error:

Train=tensorflow.data.Dataset.from_tensor_slices(([x1_train,x2_train], y_train)).batch(batch_size)
return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Can't convert non-rectangular Python sequence to Tensor.

What should I do?

Upvotes: 1

Views: 2871

Answers (2)

Mirko Bozanic Leal
Mirko Bozanic Leal

Reputation: 29

If you have a dataframe with different types (float32, int and str) you have to create it manually.

Following the Pratik's syntax:

tf.data.Dataset.from_tensor_slices(({"input_1": np.asarray(var_float).astype(np.float32), "imput_2": np.asarray(var_int).astype(np.int), ...}, labels))

Upvotes: 0

Pratik Kumar
Pratik Kumar

Reputation: 2231

If the main goal is to feed data to a model having multiple input layers then the following might be helpful:

import tensorflow as tf
from tensorflow import keras
import numpy as np

def _input_fn(n):
  x1_train = np.array([1, 2, 3, 4, 5, 6, 7, 8], dtype=np.int64)
  x2_train = np.array([15, 25, 35, 45, 55, 65, 75, 85], dtype=np.int64)

  labels = np.array([40, 30, 20, 10, 80, 70, 50, 60], dtype=np.int64)

  dataset = tf.data.Dataset.from_tensor_slices(({"input_1": x1_train, "input_2": x2_train}, labels))
  dataset = dataset.batch(2, drop_remainder=True)
  dataset = dataset.repeat(n)
  return dataset

input1 = keras.layers.Input(shape=(1,), name='input_1')
input2 = keras.layers.Input(shape=(1,), name='input_2')

model = keras.models.Model(inputs=[input_1, input_2], outputs=output)

basically instead of passing a python list, pass a dictionary where the key indicates the layer's name to which the array will be fed to.

like in the above array x1_train will be fed to tensor input1 whose name is input_1. Refered from here

Upvotes: 1

Related Questions