CODEWITHSUNDEEP
pythontensorflow
ofer-a
ofer-a

Reputation: 521

Sparse Tensor in tensorflow DataSets

I have tensorflow program that work with TFRecord and i want to read the data with tf.contrib.data.TFRecordDataset but when i try to parse the example i get an exception: "TypeError: Failed to convert object of type to Tensor" When trying with only

The code is: 

def _parse_function(example_proto):
    features = {"var_len_feature": tf.VarLenFeature(tf.float32),
                 "FixedLenFeature": tf.FixedLenFeature([10], tf.int64),
                 "label": tf.FixedLenFeature((), tf.int32default_value=0)}

parsed_features = tf.parse_single_example(example_proto, features)
return parsed_features["image"], parsed_features["label"]

filenames = ["/var/data/file1.tfrecord", "/var/data/file2.tfrecord"]
dataset = tf.contrib.data.TFRecordDataset(filenames)
dataset = dataset.map(_parse_function)

Upvotes: 4

Views: 2380

Answers (3)

ofer-a
ofer-a

Reputation: 521

TensorFlow added support for this in v1.5

https://github.com/tensorflow/tensorflow/releases/tag/v1.5.0

"tf.data now supports tf.SparseTensor components in dataset elements."

Upvotes: 2

Shahin Konadath
Shahin Konadath

Reputation: 109

tf.VarLenFeature creates SparseTensor. And most of the times the SparseTensors are associated with the mini batch. Can you try it like below?

dataset = tf.contrib.data.TFRecordDataset(filenames)

dataset = dataset.batch(batch_size=32)

dataset = dataset.map(_parse_function)

Upvotes: 0

Sebastian Porombka
Sebastian Porombka

Reputation: 1

The Tutorial in the Tensor Flow programming guide have a different indenting.

# Transforms a scalar string `example_proto` into a pair of a scalar string and
# a scalar integer, representing an image and its label, respectively.
def _parse_function(example_proto):
  features = {"image": tf.FixedLenFeature((), tf.string, default_value=""),
              "label": tf.FixedLenFeature((), tf.int32, default_value=0)}
  parsed_features = tf.parse_single_example(example_proto, features)
  return parsed_features["image"], parsed_features["label"]

# Creates a dataset that reads all of the examples from two files, and extracts
# the image and label features.
filenames = ["/var/data/file1.tfrecord", "/var/data/file2.tfrecord"]
dataset = tf.contrib.data.TFRecordDataset(filenames)
dataset = dataset.map(_parse_function)

The wrong indenting can result in a TypeError, processing an unwanted control flow by the pyton interpreter.

Upvotes: 0

Related Questions