Tensorflow placeholder declaration

Question

I'm trying to convert a tutorial from Keras to TF. I'm getting the following error:

Traceback (most recent call last):
  File "/Users/spicyramen/Documents/Development/google/python/machine_learning/deep_learning/exercise1_tf.py", line 64, in <module>
    sess.run(train_step, feed_dict=train_data)
  File "/Library/Python/2.7/site-packages/tensorflow/python/client/session.py", line 789, in run
    run_metadata_ptr)
  File "/Library/Python/2.7/site-packages/tensorflow/python/client/session.py", line 975, in _run
    % (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (768,) for Tensor u'Placeholder_1:0', which has shape '(?, 1)'

This seems to be related in how I'm passing the target labels and how my placeholder value is declared. When I return the labels I have this:

>>> dataset[:, 8].shape
(768,)
>>> dataset[:, 0:8].shape
(768, 8)

Code

import tensorflow as tf
import numpy as np

print("Tensorflow version: " + tf.__version__)
tf.set_random_seed(0)

FILENAME = 'pima-indians-diabetes.csv'
_LEARNING_RATE = 0.003
_NUM_FEATURES = 8
_NUM_LABELS = 1
_NUM_EPOCHS = 150
_BATCH_SIZE = 10


def import_data(filename):
    if filename:
        dataset = np.loadtxt(filename, delimiter=",")
        return dataset[:, 0:8], dataset[:, 8]


# create placeholder. Dataset contains _NUM_FEATURES features:
X = tf.placeholder(tf.float32, [None, _NUM_FEATURES])
Y_ = tf.placeholder(tf.float32,[None, _NUM_LABELS])  # Placeholder for correct answers

# weights and biases
W = tf.Variable(tf.random_normal([_NUM_FEATURES, _NUM_LABELS],
                                 mean=0,
                                 stddev=0.1,
                                 name='weights'))

b = tf.Variable(tf.random_normal([1, _NUM_LABELS],
                                 mean=0,
                                 stddev=0.1,
                                 name='bias'))

# activation function
Y = tf.nn.relu(tf.matmul(X, W) + b, name='activation')

# cost function i.e. sigmoid_cross_entropy_with_logits
cross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(labels=Y_, logits=Y, name='loss_function')
optimizer = tf.train.AdamOptimizer(_LEARNING_RATE)  # Formal derivation
train_step = optimizer.minimize(cross_entropy)

# cost function i.e. RMSE
# cross_entropy = tf.nn.l2_loss(Y - Y_, name="squared_error_cost")
# optimizer = tf.train.GradientDescentOptimizer(_LEARNING_RATE)
# train_step = optimizer.minimize(cross_entropy)

is_correct = tf.equal(tf.argmax(Y, 1), tf.argmax(Y_, 1))
accuracy = tf.reduce_mean(tf.cast(is_correct, tf.float32))

# init
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)

for i in range(_NUM_EPOCHS):
    # data
    batch_X, batch_Y = import_data(FILENAME)
    # train
    train_data = {X: batch_X, Y_: batch_Y}
    sess.run(train_step, feed_dict=train_data)
    a, c = sess.run([accuracy, cross_entropy], feed_dict=train_data)        
    print(str(i) + ": accuracy:" + str(a) + " loss: " + str(c))

Answer 1

This is your problem here:

>>> dataset[:, 8].shape
(768,)

TensorFlow is expecting an array of shape (768,1) and not (768,) as the error references:

Cannot feed value of shape (768,) for Tensor u'Placeholder_1:0', which has shape '(?, 1)'

The difference between the two shapes is somewhat small and Numpy would normally broadcast these for you in many circumstances, but TF won't. See the difference between those two shapes in this question with a great answer.

Luckily in your case the solution is very simple. You can use np.expand_dims() to turn your (768,) vector into a (768,1) vector, as demonstrated here:

>>> np.array([5,5,5]).shape
(3,)
>>> np.expand_dims(np.array([5,5,5]), axis=1).shape
(3, 1)

In your import_data function, simply change the return line to

return dataset[:, 0:8], np.expand_dims(dataset[:, 8], axis=1)

Edit: I like the above because np.expand_dims is a little more explicit, but there's another way that is equally simple, and others might think it's more clear---just depends on what you're used to. Figured I'd include it for completeness. The difference between a (N,) and (N,1) array is the first holds values in a 0-dimensional array np.array([5, 5, 5]) while the second holds values in a 1-dimensional array np.array([[5],[5],[5]]). You can turn your 0-d array into a 1-d array by adding a bracket around it; but then it's a row and not a column, so it needs to be transposed. So here's the two suggested ways together; B is the new suggestion, C is the above suggestion:

>>> A = np.array([5,5,5])
>>> B = np.array([A]).T
>>> C = np.expand_dims(A, axis=1)
>>> A; A.shape
array([5, 5, 5])
(3,)
>>> B; B.shape
array([[5],
       [5],
       [5]])
(3, 1)
>>> C; C.shape
array([[5],
       [5],
       [5]])
(3, 1)

Edit2: Also TensorFlow itself has a tf.expand_dims() function.