CODEWITHSUNDEEP
pythonnumpytensorflow
Nicolas Gervais
Nicolas Gervais

Reputation: 36624

FutureWarning: arrays to stack must be passed as a "sequence" type such as list or tuple. Support for non-sequence iterables is deprecated

I'm trying to turn a Tensorflow Dataset into a NumPy array and I'm getting a deprecation warning:

FutureWarning: arrays to stack must be passed as a "sequence" type such as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.

What is the "new" way of doing this?

Reproducible example with numpy 1.18.1 and tensorflow 2.1.0:

import tensorflow as tf
import numpy as np

ds = tf.data.Dataset.enumerate(tf.data.Dataset.range(20, 40))

np.vstack(ds)

What I tried: I can only do it with one dimension at a time.

np.fromiter(ds.map(lambda x, y: x), float)

Upvotes: 11

Views: 11020

Answers (2)

Tomergt45
Tomergt45

Reputation: 679

You can use tfds to do so, you can check out the documentation here.

Example how to use:

array = tfds.as_numpy(ds)

Upvotes: 0

hpaulj
hpaulj

Reputation: 231395

With just numpy, and using a generator expression:

In [105]: np.stack((np.ones(3) for _ in range(3)))                                      
/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py:3254: 
FutureWarning: arrays to stack must be passed as a "sequence" type 
such as list or tuple. Support for non-sequence iterables such as generators 
is deprecated as of NumPy 1.16 and will raise an error in the future.:
Out[105]: 
array([[1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.]])

Using a list comprehension to create the arrays:

In [106]: np.stack([np.ones(3) for _ in range(3)])                                      
Out[106]: 
array([[1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.]])

I don't use tf, so can only guess as to what:

tf.data.Dataset.enumerate(tf.data.Dataset.range(20, 40))

produces. But as I understand it tensorflow has a distinction between tensors that 'generator-like', potential executions (pipeline?), and 'eager' evaluation, in which the tensors, and tensor expressions, are actually evaluated, producing arrays (or similar object). np.stack tries to convert its inputs into arrays.

A code example for Tf.data.Dataset.enumerate:

dataset = tf.data.Dataset.from_tensor_slices([1, 2, 3])
dataset = dataset.enumerate(start=5)
for element in dataset.as_numpy_iterator():
  print(element)

enumerate returns an interator. You still have to iterate it, as in this for loop. OR list(dataset).

https://www.tensorflow.org/api_docs/python/tf/data/Dataset#enumerate

Upvotes: 3

Related Questions