CODEWITHSUNDEEP
pythontensorflow
David Parks
David Parks

Reputation: 32051

How do you feed a tf.data.Dataset dynamically in eager execution mode where initializable_iterator isn't available?

What is the new approach (under eager execution) to feeding data through a dataset pipeline in a dynamic fashion, when we need to feed it sample by sample?

I have a tf.data.Dataset which performs some preprocessing steps and reads data from a generator, drawing from a large dataset during training.

Let's say that dataset is represented as:

ds = tf.data.Dataset.from_tensor_slices([1, 2, 3, 4, 5, 6])
ds = ds.map(tf.square).shuffle(2).batch(2)
iterator = tf.data.make_one_shot_iterator(ds)

After training I want to produce various visualizations which require that I feed one sample at a time through the network for inference. I've now got this dataset preprocessing pipeline that I need to feed my raw sample through to be sized and shaped appropriately for the network input.

This seems like a use case for the initializable iterator:

placeholder = tf.placeholder(tf.float32, shape=None)
ds = tf.data.Dataset.from_tensor_slices(placeholder)
ds = ds.map(tf.square).shuffle(2).batch(2)
iterator = tf.data.make_initializable_iterator(ds) 
# now re-initialize for each sample

Keep in mind that the map operation in this example represents a long sequence of preprocessing operations that can't be duplicated for each new data sample being feed in.

This doesn't work with eager execution, you can't use the placeholder. The documentation examples all seem to assume a static input such as in the first example here.

The only way I can think of doing this is with a queue and tf.data.Dataset.from_generator(...) which reads from the queue that I push to before predicting on the data. But this feels both hacky, and appears prone to deadlocks that I've yet to solve.

TF 1.14.0

Upvotes: 3

Views: 1128

Answers (1)

David Parks
David Parks

Reputation: 32051

I just realized that the answer to this question is trivial:

Just create a new dataset!

In non-eager mode the code below would have degraded in performance because each dataset operation would have been added to the graph and never released, and in non-eager mode we have the initializable iterator to resolve that issue.

However, in eager execution mode tensorflow operations like this are ephemeral, added iterators aren't being added to a global graph, they just get created and die when no longer referenced. Win one for TF2.0!

The code below (copy/paste runnable) demonstrates:

import tensorflow as tf
import numpy as np
import time


tf.enable_eager_execution()

inp = np.ones(shape=5000, dtype=np.float32)

t = time.time()
while True:
    ds = tf.data.Dataset.from_tensors(inp).batch(1)
    val = next(iter(ds))
    assert np.all(np.squeeze(val, axis=0) == inp)
    print('Processing time {:.2f}'.format(time.time() - t))
    t = time.time()

The motivation for the question came on the heels of this issue in 1.14 where creating multiple dataset operations in graph mode under Keras constitutes a memory leak.

https://github.com/tensorflow/tensorflow/issues/30448

Upvotes: 1

Related Questions