CODEWITHSUNDEEP
pythontensorflowkerastensorflow2.0
salouri
salouri

Reputation: 821

How to use tensorflow.data.Dataset.flat_map(my_func) with a function of multiple paramters?

I have this code as part of windowing function:

    dataset = tf.data.Dataset.from_tensor_slices(series)
    dataset = dataset.window(size=window_size, shift=1, drop_remainder=True)
    dataset = dataset.flat_map(lambda window: window.batch(window_size))

However, in the TF Certification exam I can't use lambda! so, I need to get rid of the lambda part. I tried using a function like so:

def window_to_tensor(window_ds: tf.data.Dataset, window_size: int):
    return window_ds.batch(window_size)

dataset = dataset.flat_map(window_to_tensor ,window_size)

But I get an error:

TypeError: flat_map() takes 2 positional arguments but 3 were given

Even when I try to use two functions like below, I get a similar error:

dataset = dataset.apply(window_to_tensor,window_size)
dataset = tf.keras.backend.flatten(dataset)

How can I map a function then flatten the resulting dataset without using flat_map (or with using it but without lambda)? (I am required to use TF2.0.0)

Upvotes: 0

Views: 708

Answers (1)

Nicolas Gervais
Nicolas Gervais

Reputation: 36624

You can use lambda. Only lambda layers are forbidden:

enter image description here

You can read the instructions yourself.

So you can do the windowed dataset like this:

import tensorflow as tf
import numpy as np

window_size = 5

dataset = tf.data.Dataset.from_tensor_slices(np.random.rand(100))
dataset = dataset.window(window_size, shift=1, drop_remainder=True)
dataset = dataset.flat_map(lambda window: window.batch(5))
dataset = dataset.map(lambda window: (window[:-1], window[-1:])).batch(4)

next(iter(dataset))

(<tf.Tensor: shape=(4, 4), dtype=float64, numpy=
 array([[0.82421497, 0.02775336, 0.51822687, 0.99682813],
        [0.02775336, 0.51822687, 0.99682813, 0.25681553],
        [0.51822687, 0.99682813, 0.25681553, 0.80613281],
        [0.99682813, 0.25681553, 0.80613281, 0.01170842]])>,
 <tf.Tensor: shape=(4, 1), dtype=float64, numpy=
 array([[0.25681553],
        [0.80613281],
        [0.01170842],
        [0.01685387]])>)

Upvotes: 1

Related Questions