CODEWITHSUNDEEP
pythontensorflow
KOB
KOB

Reputation: 4545

Using the value of of a Tensor as the shape of another?

I have the following two lines in my code:

numSequences = tf.placeholder(tf.float32, shape=())
...
prediction = tf.reshape(predictionFlat, [numSequences, sequenceLength, vocabSize])

Is it possible to extract the scalar value out of the numSequences tensor to use it as a value when defining the shape of the prediction tensor?

EDIT

Here is more of my code:

x = tf.placeholder(tf.float32, [None, sequenceLength, vocabSize])
y = tf.placeholder(tf.float32, [None, sequenceLength, vocabSize])
numSequences = tf.placeholder(tf.float32, shape=())
xFlat = tf.contrib.layers.flatten(x)    # [batchSize, sequenceLength*vocabSize]

W = tf.Variable(tf.random_normal([hiddenDimension, sequenceLength, vocabSize]))
b = tf.Variable(tf.random_normal([1, sequenceLength, vocabSize]))
WFlat = tf.contrib.layers.flatten(W)
bFlat = tf.contrib.layers.flatten(b)

cell = rnn.BasicLSTMCell(hiddenDimension, forget_bias=forgetRate)
outputs, states = tf.nn.static_rnn(cell, [xFlat], dtype=tf.float32) 
predictionFlat = tf.add(tf.matmul(outputs[0], WFlat), bFlat) # outputs = [np.array([batchSize, hiddenDimension])] -> outputs[0] = [batchSize, hiddenDimension]
prediction = tf.reshape(predictionFlat, [numSequences, sequenceLength, vocabSize])

EDIT 2

I am attempting to do something similar where I need my sequenceLength variable (a parameter of the shapes of my tensors) to be a placeholder rather than a fixed value. I have implemented this the same way I did for numSequences, but I am getting the error shown below. I am failing to understand how this is different to the previous numSequences implementation I originally asked about.

Code:

numSequences = tf.placeholder(tf.int32, shape=())
seqLength = tf.placeholder(tf.int32, shape=())
x = tf.placeholder(tf.float32, [None, seqLength, vocabSize])
y = tf.placeholder(tf.float32, [None, seqLength, vocabSize])
xFlat = tf.contrib.layers.flatten(x)                                                # [batchSize, sequenceLength*vocabSize]

W = tf.Variable(tf.random_normal([hiddenDimension, seqLength, vocabSize]))
b = tf.Variable(tf.random_normal([1, seqLength, vocabSize]))
WFlat = tf.contrib.layers.flatten(W)                                                # [hiddenDimension, sequenceLength*vocabSize]
bFlat = tf.contrib.layers.flatten(b)                                                # [1, sequenceLength*vocabSize]

cell = rnn.BasicLSTMCell(hiddenDimension, forget_bias=forgetRate)
outputs, states = tf.nn.static_rnn(cell, [xFlat], dtype=tf.float32)                 # outputs    = [[batchSize, hiddenDimension]]
predictionFlat = tf.add(tf.matmul(outputs[0], WFlat), bFlat)                        # outputs[0] = [batchSize, hiddenDimension]
prediction = tf.reshape(predictionFlat, [numSequences, seqLength, vocabSize])

Error:

    x = tf.placeholder(tf.float32, [None, seqLength, vocabSize])
  File "/usr/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1530, in placeholder
    return gen_array_ops._placeholder(dtype=dtype, shape=shape, name=name)
  File "/usr/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1954, in _placeholder
    name=name)
  File "/usr/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 705, in apply_op
    attr_value.shape.CopyFrom(_MakeShape(value, key))
  File "/usr/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 198, in _MakeShape
    return tensor_shape.as_shape(v).as_proto()
  File "/usr/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 798, in as_shape
    return TensorShape(shape)
  File "/usr/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 434, in __init__
    self._dims = [as_dimension(d) for d in dims_iter]
  File "/usr/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 434, in <listcomp>
    self._dims = [as_dimension(d) for d in dims_iter]
  File "/usr/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 376, in as_dimension
    return Dimension(value)
  File "/usr/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 32, in __init__
    self._value = int(value)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'Tensor'

Upvotes: 2

Views: 783

Answers (1)

P-Gn
P-Gn

Reputation: 24581

Yes, tensor shapes can usually be tensors themselves, however they need to be of integer type.

import tensorflow as tf

x = tf.constant([2, 3], dtype=tf.int32)
y = tf.zeros((x[0], x[1], 5))

sess = tf.InteractiveSession()
print(y.eval().shape)
# (2, 3, 5)

EDIT

Closer to your example,

import tensorflow as tf

x = tf.placeholder(tf.int32, shape=())
y = tf.zeros((500,))

sess = tf.InteractiveSession()
print(sess.run(tf.shape(tf.reshape(y, [x, x, 5])), {x: 10}))
# [10 10 5]

Of course, reshaping should preserve the total number of elements in y, so it has only a limited usefulness.

EDIT 2

You cannot parameterize the shape of one placeholder by another placehoder. It does not make sense since placeholders are provided at the same time. Provide a placeholder with unknown shape then provide a parameterized reshape op as you did in your first example.

Upvotes: 2

Related Questions