Tensorflow returns too many predictions

Question

I am working on a TensorFlow CNN Model for mnist, modifying this example: https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3_NeuralNetworks/convolutional_network_raw.py.

When running the testing 256 mnist images, it returns 784 predictions instead of 256. I am guessing the 784 comes from the mnist image size (28 pixels x 28 pixels = 784), however I am unclear where the axis alignment could be going wrong, if it is in fact an axis alignment issue.

Specifically I get the following error from this line in the code correct_pred = tf.equal(tf.argmax(prediction, 1), tf.argmax(Y, 1)):

InvalidArgumentError (see above for traceback): Incompatible shapes: [784] vs. [256]
     [[Node: Equal = Equal[T=DT_INT64, _device="/job:localhost/replica:0/task:0/device:CPU:0"](ArgMax, ArgMax_1)]]

Where things may be going wrong:

• My weight and bias matrix dimensions may not be aligning like I expect. I am attempting to use manually defined weights/biases, and while I think the dimensions align, I am clearly missing something.

• I may not be using the dense layer correctly. I added the dense layer to go from the weights (4x4x50x500) to the correct size for the classification layer (500x10). The image below shows debugging info and these dimensions.

  

Code

import tensorflow as tf
import pickle

input_size = 28  # e.g. 28x28 input
model = pickle.load(open("model.p", "rb" ))

weights = {
    'wc1': tf.Variable(model[0]['weights']),  # 5x5x20
    'wc2': tf.Variable(model[2]['weights']),  # 5x5x20x50
    'wd1': tf.Variable(model[4]['weights']),  # 4x4x50x500
    'out': tf.Variable(model[5]['weights'])  # 500x10
}

biases = {
    'bc1': tf.Variable(model[0]['bias']),  # 20
    'bc2': tf.Variable(model[2]['bias']),  # 50
    'bd1': tf.Variable(model[4]['bias']),  # 500
    'out': tf.Variable(model[5]['bias']),  # 10
    }


# Import MNIST data
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("./MNIST-data/", one_hot=True)

# tf Graph input
X = tf.placeholder(tf.float32, [None, 784])
Y = tf.placeholder(tf.float32, [None, 10])
keep_prob = tf.placeholder(tf.float32)  # dropout (keep probability)



# Create some wrappers for simplicity
def conv2d(x, W, b, strides=1):
    # Conv2D wrapper, with bias and relu activation
    x = tf.nn.conv2d(x, W,strides=[1, strides, strides, 1], padding='SAME')
    x = tf.nn.bias_add(x, b)
    return tf.nn.relu(x)

def maxpool2d(x, k=2):
    # MaxPool2D wrapper
    return tf.nn.max_pool(x, ksize=[1, k, k, 1], strides=[1, k, k, 1],
                          padding='SAME')

# Create model
def conv_net(x, weights, biases):
    x = tf.reshape(x, shape=[-1, input_size, input_size, 1])
    conv1 = conv2d(x, tf.reshape(weights['wc1'], shape=[5, 5, 1, 20]), biases['bc1'])
    pool1 = maxpool2d(conv1, k=2)
    conv2 = conv2d(pool1, weights['wc2'], biases['bc2'])
    pool2 = maxpool2d(conv2, k=2)

    # Fully connected layer
    # Reshape conv2 output to fit fully connected layer input
    pool2_flat = tf.reshape(pool2, [-1, 4 * 4 * 50])
    dense = tf.layers.dense(inputs=pool2_flat, units=500, activation=tf.nn.relu)

    # Output, class prediction
    out = tf.add(tf.matmul(dense, weights['out']), biases['out'])  # shape = (?, 10)
    return out


logits = conv_net(X, weights, biases)
prediction = tf.nn.softmax(logits)

# # Evaluate model
correct_pred = tf.equal(tf.argmax(prediction, 1), tf.argmax(Y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))

init = tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init)
    # Calculate accuracy for 256 MNIST test images
    print("Testing Accuracy:", \
          sess.run(accuracy, {X: mnist.test.images[:256], Y: mnist.test.labels[:256], keep_prob: 1.0}))

Answer 1

UPDATE: I switched to tensorflow slim and all my problems were pretty much instantaneously solved. I never figured out the issue with the above code, so this isn't technically an answer, just wanted to provide advice if anybody else has similar problems.