Cannot feed value of shape: Misunderstanding Tensor Dimensions

Question

I've been walking through some tensorflow tutorials and am cobbling together a pet experiment. However, I am running into some dimension errors and I can seem to figure them out.

My goal: I have an input matrix for the shape 1xN. I have a training set of dimension 10xN. (1 and 10 were chosen arbitrarily). N is intended to represent N samples in a training set: 1 input value mapped to one vector of outputs. You can think of this as 1 input neuron and m output neurons. The training set is a set of these single values mapped to a 1d vector. I wish to train the network by running the set of these mapped inputs and outputs against it and reducing the error.

The simple algorithm that I am trying to accomplish:

1. For each value in the input vector
   2. Load the input neuron with that value
   3. Feed forward
   4. Evaluate against the corresponding vector

Repeat to minimize error.

However, I seem to be getting mixed up with how to format the data to feed to the network. I have a placeholder of 1 input neurons and one of n output neurons. I want to follow the above algorithm but I am not sure if I am doing it right:

# Data parameters

num_frames = 10

stimuli_value_low = .00001
stimuli_value_high = 100

pixel_value_low = .00001
pixel_value_high = 256.0

stimuli_dimension = 1
frame_dimension = 10

stimuli = np.random.uniform(stimuli_value_low, stimuli_value_high, (stimuli_dimension, num_frames))
frames = np.random.uniform(pixel_value_low, pixel_value_high, (frame_dimension, num_frames))

# Parameters
learning_rate = 0.01
training_iterations = 1000
display_iteration = 10

# Network Parameters
n_hidden_1 = 100
n_hidden_2 = 100
num_input_neurons = stimuli_dimension
num_output_neurons = frame_dimension

# Create placeholders
input_placeholder = tf.placeholder("float", [None, num_input_neurons])
output_placeholder = tf.placeholder("float", [None, num_output_neurons])

# Store layers weight & bias
weights = {
    'h1': tf.Variable(tf.random_normal([num_input_neurons, n_hidden_1])),
    'h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),
    'out': tf.Variable(tf.random_normal([n_hidden_2, num_output_neurons]))
}
biases = {
    'b1': tf.Variable(tf.random_normal([n_hidden_1])),
    'b2': tf.Variable(tf.random_normal([n_hidden_2])),
    'out': tf.Variable(tf.random_normal([num_output_neurons]))
}

# Create model
def neural_net(input_placeholder):
    # Hidden fully connected layer 
    layer_1 = tf.add(tf.matmul(input_placeholder, weights['h1']), biases['b1'])
    # Hidden fully connected layer
    layer_2 = tf.add(tf.matmul(layer_1, weights['h2']), biases['b2'])
    # Output fully connected layer with a neuron for each pixel
    out_layer = tf.matmul(layer_2, weights['out']) + biases['out']
    return out_layer

# Construct model
logits = neural_net(input_placeholder)

# Define loss operation and optimizer
loss_operation = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = logits, labels = output_placeholder))
optimizer = tf.train.AdamOptimizer(learning_rate = learning_rate)
train_operation = optimizer.minimize(loss_operation)

# Evaluate model (with test logits, for dropout to be disabled)
correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(output_placeholder, 1))
accuracy_operation = tf.reduce_mean(tf.cast(correct_pred, tf.float32))

# Initialize the variables (i.e. assign their default value)
init = tf.global_variables_initializer()

# Start Training
with tf.Session() as sess:

  # Run the initializer
  sess.run(init)

  for step in range(1, training_iterations + 1): 

    sess.run(train_operation, feed_dict = {X: stimuli, Y: frames})

    if iteration % display_iteration == 0 or iteration == 1:

      loss, accuracy = sess.run([loss_operation, accuracy_operation], feed_dict = {X: stimuli, Y: frames})

      print("Step " + str(iteration) + 
            ", Loss = " + "{:.4f}".format(loss) + 
            ", Training Accuracy= " + \
            "{:.3f}".format(acc))

  print("Optimization finished!")

I think it is something to do with how I am structuring my data or feeding it to the run function.

Here is the error I am getting:

ValueError                                Traceback (most recent call last)

<ipython-input-420-7517598734d6> in <module>()
      6   for step in range(1, training_iterations + 1):
      7 
----> 8     sess.run(train_operation, feed_dict = {X: stimuli, Y: frames})
      9 
     10     if iteration % display_iteration == 0 or iteration == 1:

1 frames

/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
   1147                              'which has shape %r' %
   1148                              (np_val.shape, subfeed_t.name,
-> 1149                               str(subfeed_t.get_shape())))
   1150           if not self.graph.is_feedable(subfeed_t):
   1151             raise ValueError('Tensor %s may not be fed.' % subfeed_t)

ValueError: Cannot feed value of shape (1, 10) for Tensor 'Placeholder_6:0', which has shape '(?, 1)'

How can I ensure I am formatting my input data correctly and forming my network corresponingly?

Answer 1

Turns out I had the dimensions of the arrays I was generating backwards:

stimuli = np.random.uniform(stimuli_value_low, stimuli_value_high, (stimuli_dimension, num_frames))
frames = np.random.uniform(pixel_value_low, pixel_value_high, (frame_dimension, num_frames))

should be:

stimuli = np.random.uniform(stimuli_value_low, stimuli_value_high, (num_frames, stimuli_dimension))
frames = np.random.uniform(pixel_value_low, pixel_value_high, (num_frames, frame_dimension))