CODEWITHSUNDEEP
tensorflow
Ron Cohen
Ron Cohen

Reputation: 2925

Tensorflow single-value vs. batch tensors

One of Tensorflow's mysteries for me is: when does a variable represent a single tensor vs. a batch? Below is an example of a network with a continuous value output (not a classifier). In the line beginning with loss I calculate the loss by subtracting predictions from truth values and summing the absolute values of those differences over a batch. In that line, Tensorflow inserts a batch of truth values for truthValues_placeholder, so truthValues_placeholder is a "batch" tensor, i.e. it has one entry for each item in the batch. However, the previous line calculates prediction as a single value (as opposed to a batch). My question: Am I correct that Tensorflow is magically changes prediction into a "batch tensor", so it also has one entry calculated for each item in the batch?

graph = tf.Graph()
with tf.Graph().as_default():
    ...
    # Network layers here
    ...
    # Final layer
    prediction = tf.matmul(inputsToLastLayer, weightsOutputLayer) + biasesOutputLayer
    loss = tf.nn.l2_loss(tf.sub(prediction, truthValues_placeholder))
    ...
    global_step = tf.Variable(0, name='global_step', trainable=False)
    train_op = optimizer.minimize(loss, global_step=global_step)
    ...
    sess = tf.Session()
    init = tf.initialize_all_variables()
    sess.run(init)
    for step in xrange(maxSteps):
        feed_dict = fill_feed_dict(..., truthValues_placeholder,...)

Upvotes: 1

Views: 905

Answers (1)

Jules Gagnon-Marchand
Jules Gagnon-Marchand

Reputation: 3801

Tensorflow behaves the same way as numpy, in that it broadcasts basic operations with individual units to the whole matrix, like

In [3]: np.zeros([10, 10]) - 1
Out[3]: 
array([[-1., -1., -1., -1., -1., -1., -1., -1., -1., -1.],
   [-1., -1., -1., -1., -1., -1., -1., -1., -1., -1.],
   [-1., -1., -1., -1., -1., -1., -1., -1., -1., -1.],
   [-1., -1., -1., -1., -1., -1., -1., -1., -1., -1.],
   [-1., -1., -1., -1., -1., -1., -1., -1., -1., -1.],
   [-1., -1., -1., -1., -1., -1., -1., -1., -1., -1.],
   [-1., -1., -1., -1., -1., -1., -1., -1., -1., -1.],
   [-1., -1., -1., -1., -1., -1., -1., -1., -1., -1.],
   [-1., -1., -1., -1., -1., -1., -1., -1., -1., -1.],
   [-1., -1., -1., -1., -1., -1., -1., -1., -1., -1.]])

So, nothing really surprising. You can test your operations with NumPy first if you want to be sure.

Upvotes: 1

Related Questions