Reputation: 33
i'm trying to build a data pipelines from tfrecords, here is my code
def _parse_image_function(example_proto):
keys_to_features = {
'image/encoded': tf.io.FixedLenFeature((), tf.string),
'image/format': tf.io.FixedLenFeature((), tf.string, default_value='jpeg'),
'image/height': tf.io.FixedLenFeature([1], tf.int64),
'image/width': tf.io.FixedLenFeature([1], tf.int64),
'image/channels': tf.io.FixedLenFeature([1], tf.int64),
'image/shape': tf.io.FixedLenFeature([3], tf.int64),
'image/object/bbox/xmin': tf.io.VarLenFeature(dtype=tf.float32),
'image/object/bbox/ymin': tf.io.VarLenFeature(dtype=tf.float32),
'image/object/bbox/xmax': tf.io.VarLenFeature(dtype=tf.float32),
'image/object/bbox/ymax': tf.io.VarLenFeature(dtype=tf.float32),
'image/object/bbox/label': tf.io.VarLenFeature(dtype=tf.int64),
'image/object/bbox/difficult': tf.io.VarLenFeature(dtype=tf.int64),
'image/object/bbox/truncated': tf.io.VarLenFeature(dtype=tf.int64),
}
example = tf.io.parse_single_example(example_proto, keys_to_features)
image = tf.io.decode_raw(example['image/encoded'], tf.int32)
return image
Then, i get image after decode
for img in train_ds:
print(img.numpy())
But i got an error
tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __inference_Dataset_map__parse_image_function_78}} Input to DecodeRaw has length 286478 that is not a multiple of 4, the size of int32
[[{{node DecodeRaw}}]] [Op:IteratorGetNextSync]
How could I fix it?
Upvotes: 0
Views: 681
Reputation: 109
As mentioned in the comments, the error message is about decoding error, and not a problem with iteration. You are creating the dataset object and iterating through it correctly.
image = tf.io.decode_raw(example['image/encoded'], tf.int32)
tells TensorFlow to decode the data stored in that key as a tensor of int32
s. That is, raw binary values of 32-bit integers, for example, the contents of .data
in a NumPy array with dtype=np.int32
.
Since you have read in the binary contents of .jpg
files, I'm assuming that you have binaries of JPEG images at that key value. For this case, you should be using the decode_jpeg
method instead. You should be using:
image = tf.io.decode_jpeg(example['image/encoded'])
.
decode_jpeg
also provides you with some options on how you wish the JPEG data to be decoded (e.g. grayscale only, chroma upsampling method). Full documentation of decode_jpeg
is available here
Additionally, TensorFlow also provides a decode_image
that automatically calls the correct supported image format decoder given the binary data. See docs here.
Upvotes: 1