CODEWITHSUNDEEP
pythontensorflow
Rohit
Rohit

Reputation: 10296

Trying to read csv dataset into tensorflow

I have csv files(train and test) that contains data in the following format

enter image description here

I am trying to load my data (100MB) into tensorflow input pipeline (not sure if this is the right way to do it). 

  x, y = tf.placeholder(tf.float32, shape=[None,2]), tf.placeholder(tf.float32, shape=[None,1])
 dataset = tf.data.Dataset.from_tensor_slices((x, y))

train_dataset = tf.contrib.data.make_csv_dataset(csv_path+'\csvdatatrain.csv', batch_size=32)
test_dataset = tf.contrib.data.make_csv_dataset(csv_path+'\csvdatatest.csv', batch_size=32)


iter = dataset.make_initializable_iterator()
features, labels = iter.get_next()
with tf.Session() as sess:
    #initialise iterator with train data
    sess.run(iter.initializer, feed_dict={x: train_dataset[0], y: train_dataset[1]})

    for _ in range(EPOCHS):
        #### Training
        print('This is training')

        #### Testing
        sess.run(iter.initializer, feed_dict={ x: test_dataset[0], y: test_dataset[1]})
        print('This is testing')

I am getting an error while initializing iterator with train data

sess.run(iter.initializer, feed_dict={ x: train_dataset1, y: train_dataset [0]}) TypeError: 'PrefetchDataset' object does not support indexing

Upvotes: 0

Views: 1451

Answers (1)

Ehab AlBadawy
Ehab AlBadawy

Reputation: 3485

You don't need a placeholder if you are gonna use tensorflow data loader. You either load the csv file with numpy and pass it to the placeholders through feed_dict or use tensorflow data loader and only pass the path to csv file.

If you're gonna use tensorflow data loader, you don't need to init the iterator every epoch! it is only initialized one time and no feed_dict is needed there.

This code works with me on tensorflow 1.8 (used iris dataset there)

EPOCHS = 10
dataset = tf.contrib.data.make_csv_dataset('iris.csv',batch_size=1)
iterator = dataset.make_initializable_iterator()
next_elemnet = iterator.get_next()
with tf.Session() as sess:
    sess.run(iterator.initializer)
    for _ in range(EPOCHS):
        print(sess.run(next_elemnet))

The output should look like so:

{'sepal_length': array([ 6.4000001], dtype=float32), 'sepal_width': array([ 2.79999995], dtype=float32), 'petal_length': array([ 5.5999999], dtype=float32), 'petal_width': array([ 2.20000005], dtype=float32), 'species': array([b'virginica'], dtype=object)}
{'sepal_length': array([ 5.19999981], dtype=float32), 'sepal_width': array([ 2.70000005], dtype=float32), 'petal_length': array([ 3.9000001], dtype=float32), 'petal_width': array([ 1.39999998], dtype=float32), 'species': array([b'versicolor'], dtype=object)}
{'sepal_length': array([ 4.80000019], dtype=float32), 'sepal_width': array([ 3.4000001], dtype=float32), 'petal_length': array([ 1.89999998], dtype=float32), 'petal_width': array([ 0.2], dtype=float32), 'species': array([b'setosa'], dtype=object)}
{'sepal_length': array([ 7.69999981], dtype=float32), 'sepal_width': array([ 2.79999995], dtype=float32), 'petal_length': array([ 6.69999981], dtype=float32), 'petal_width': array([ 2.], dtype=float32), 'species': array([b'virginica'], dtype=object)}
{'sepal_length': array([ 4.4000001], dtype=float32), 'sepal_width': array([ 3.], dtype=float32), 'petal_length': array([ 1.29999995], dtype=float32), 'petal_width': array([ 0.2], dtype=float32), 'species': array([b'setosa'], dtype=object)}
{'sepal_length': array([ 5.], dtype=float32), 'sepal_width': array([ 3.], dtype=float32), 'petal_length': array([ 1.60000002], dtype=float32), 'petal_width': array([ 0.2], dtype=float32), 'species': array([b'setosa'], dtype=object)}
{'sepal_length': array([ 6.4000001], dtype=float32), 'sepal_width': array([ 3.20000005], dtype=float32), 'petal_length': array([ 5.30000019], dtype=float32), 'petal_width': array([ 2.29999995], dtype=float32), 'species': array([b'virginica'], dtype=object)}
{'sepal_length': array([ 4.5], dtype=float32), 'sepal_width': array([ 2.29999995], dtype=float32), 'petal_length': array([ 1.29999995], dtype=float32), 'petal_width': array([ 0.30000001], dtype=float32), 'species': array([b'setosa'], dtype=object)}
{'sepal_length': array([ 5.], dtype=float32), 'sepal_width': array([ 3.5], dtype=float32), 'petal_length': array([ 1.60000002], dtype=float32), 'petal_width': array([ 0.60000002], dtype=float32), 'species': array([b'setosa'], dtype=object)}
{'sepal_length': array([ 6.], dtype=float32), 'sepal_width': array([ 2.20000005], dtype=float32), 'petal_length': array([ 5.], dtype=float32), 'petal_width': array([ 1.5], dtype=float32), 'species': array([b'virginica'], dtype=object)}

Upvotes: 1

Related Questions