CODEWITHSUNDEEP
pythontensorflowdatasettensorflow2.0tensorflow-datasets
Vivian
Vivian

Reputation: 355

TensorFlow: convert tf.Dataset to tf.Tensor

I want to generate windows of the range of 10:

import tensorflow as tf

dataset = tf.data.Dataset.from_tensor_slices(tf.range(10))
dataset = dataset.window(5, shift=1, drop_remainder=True)

and would like to train my model on this dataset.

To do so, those windows have to be converted to tensors. But the datatype of these windows cannot be converted via tf.convert_to_tensor to a tensor. It is possible to do tf.convert_to_tensor(list(window)) but this is quite inefficient.

Does anyone know how to convert a tf.VariantDataset efficiently to a tf.Tensor?

Thank you for your help!

Upvotes: 5

Views: 2501

Answers (1)

javidcf
javidcf

Reputation: 59731

If you want to create a tensor of sliding windows, doing it through a dataset is not really the best way, is far less efficient and flexible. I don't think there is a proper operation for that, but there are two similar ones for 2D and 3D arrays, tf.image.extract_patches and tf.extract_volume_patches. You can reshape your 1D data to use them:

import tensorflow as tf

a = tf.range(10)
win_size = 5
stride = 1
# Option 1
a_win = tf.image.extract_patches(tf.reshape(a, [1, -1, 1, 1]),
                                 sizes=[1, win_size, 1, 1],
                                 strides=[1, stride, 1, 1],
                                 rates=[1, 1, 1, 1],
                                 padding='VALID')[0, :, 0]
# Option 2
a_win = tf.extract_volume_patches(tf.reshape(a, [1, -1, 1, 1, 1]),
                                  ksizes=[1, win_size, 1, 1, 1],
                                  strides=[1, stride, 1, 1, 1],
                                  padding='VALID')[0, :, 0, 0]
# Print result
print(a_win.numpy())
# [[0 1 2 3 4]
#  [1 2 3 4 5]
#  [2 3 4 5 6]
#  [3 4 5 6 7]
#  [4 5 6 7 8]
#  [5 6 7 8 9]]

Upvotes: 2

Related Questions