CODEWITHSUNDEEP
pythontensorflowneural-network
newcomer
newcomer

Reputation: 3

Neural Network - ValueError: Cannot feed value of shape

I'm new in Python and Tensorflow . For the beginning I watched the MNIST tutorial and understood it so far.

But now I have to create a new Neural Network with numerical input_datas. I got a dataset which delivers an input_data and v_data.

If I run input_data.shape -> (1000,25,4)

If I run v_data.shape -> (1000,2)

What I tried to do is to split the data for (Training + Validation) and Testing.

And then I devided the 90% of the input_data in training and validation (70% training, 30% validation)

The network should correctly predict based on v_data, but I still get an error. See the code and the error below.

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

# Imports
import tensorflow as tf
import pickle as pkl
import numpy as np

# load data
with open('input.pkl', 'rb') as f:
    input_data = pkl.load(f)
    f.close()

X_train, y_train = input_data


#split data into train / validation and test
X_input = X_train[0:900]
y_input = y_train[0:900]

#print (X_input.shape)
#print (y_input.shape)

X_train_data = X_input[0:630]
X_test_data = X_input[630:900]

y_train_data = y_input[0:630]
y_test_data = y_input[630:900]

# Variables
hidden_layer_1_nodes = 300
hidden_layer_2_nodes = 100
output_layer_nodes = 100
epochs = 10
classes = 2
epoch_errors = []
stddev = 0.035
learning_rate = 0.08
batch_size = 100

#print (X_train_data[0])

# TF Placeholders
X = tf.placeholder('float', [25, 4], name='X')
y = tf.placeholder('float', name='y')

# Weights Matrices
W1 = tf.Variable(tf.truncated_normal([4, hidden_layer_1_nodes], stddev=stddev), name='W1')
W2 = tf.Variable(tf.truncated_normal([hidden_layer_1_nodes, hidden_layer_2_nodes], stddev=stddev), name='W2')
W3 = tf.Variable(tf.truncated_normal([hidden_layer_2_nodes, output_layer_nodes], stddev=stddev), name='W3')
W4 = tf.Variable(tf.truncated_normal([output_layer_nodes, classes], stddev=stddev), name='W4')

# Biases Vectors
b1 = tf.Variable(tf.truncated_normal([hidden_layer_1_nodes], stddev=stddev), name='b1')
b2 = tf.Variable(tf.truncated_normal([hidden_layer_2_nodes], stddev=stddev), name='b2')
b3 = tf.Variable(tf.truncated_normal([output_layer_nodes], stddev=stddev), name='b3')
b4 = tf.Variable(tf.truncated_normal([classes], stddev=stddev), name='b4')

# Define the Neural Network
def nn_model(X):
    input_layer     =    {'weights': W1, 'biases': b1}
    hidden_layer_1  =    {'weights': W2, 'biases': b2}
    hidden_layer_2  =    {'weights': W3, 'biases': b3}
    output_layer    =    {'weights': W4, 'biases': b4}

    input_layer_sum = tf.add(tf.matmul(X, input_layer['weights']), input_layer['biases'])
    input_layer_sum = tf.nn.relu(input_layer_sum)

    hidden_layer_1_sum = tf.add(tf.matmul(input_layer_sum, hidden_layer_1['weights']), hidden_layer_1['biases'])
    hidden_layer_1_sum = tf.nn.relu(hidden_layer_1_sum)

    hidden_layer_2_sum = tf.add(tf.matmul(hidden_layer_1_sum, hidden_layer_2['weights']), hidden_layer_2['biases'])
    hidden_layer_2_sum = tf.nn.relu(hidden_layer_2_sum)

    output_layer_sum = tf.add(tf.matmul(hidden_layer_2_sum, output_layer['weights']), output_layer['biases'])
    return output_layer_sum

# Train the Neural Network
def nn_train(X):
    pred = nn_model(X)
    pred = tf.identity(pred)
    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y))
    optimizer = tf.train.AdagradOptimizer(learning_rate=learning_rate).minimize(cost)

    with tf.Session() as sess:
        init_op = tf.global_variables_initializer()
        #saver = tf.train.Saver()
        sess.run(init_op)

        for epoch in range(epochs):
            epoch_loss = 0.0

            i = 0
            while i < len(X_train_data):
                start = i
                end = i+batch_size

                batch_x = np.array(X_train_data[start:end])
                batch_y = np.array(y_train_data[start:end])

                _, c = sess.run([optimizer, cost], feed_dict={X: batch_x, y: batch_y})
                epoch_loss += c
                i+= batch_size

            epoch_errors.append(epoch_loss)
            print('Epoch ', epoch + 1, ' of ', epochs, ' with loss: ', epoch_loss)

        correct_result = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
        accuracy = tf.reduce_mean(tf.cast(correct_result, 'float'))
        print('Acc: ', accuracy.eval({X:X_test_data, y:y_test_data}))

if __name__ == "__main__":
    nn_train(X)

The following error

ValueError: Cannot feed value of shape (100, 25, 4) for Tensor 'X:0', which has shape '(25, 4)'

occurs in line 105

_, c = sess.run([optimizer, cost], feed_dict={X: batch_x, y: batch_y})

Upvotes: 0

Views: 362

Answers (2)

Kumar
Kumar

Reputation: 776

The placeholder you have defined for the input X has the shape (25,4)

tf.placeholder('float', [25, 4], name='X')

But the input you are providing is of the shape (100, 25, 4) where 100 is your batch size. Change the definition to 

tf.placeholder('float', [None, 25, 4], name='X')

and the error should be gone. Here, None takes care of batch size, automatically.

Update: Sorry, I didn't go through the whole code. You code needs a few fixes. The correct syntax for feeding data to a placeholder is:

X = tf.placeholder(tf.float32, [None, input_dim], name='X')

Now, if you are dealing with the images, your input_dim will be the length of the flattened array for one example i.e. If your image has dimension 25x4, the input_dim should be 25*4=100. It should be equal to the first dimension of your weight layer 1 here W1. Also, before feeding your batch you will need to reshape it.

Below is the fixed code(changes are commented):

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

# Imports
import tensorflow as tf
import pickle as pkl
import numpy as np

# load data
with open('input.pkl', 'rb') as f:
    input_data = pkl.load(f)
    f.close()

X_train, y_train = input_data


#split data into train / validation and test
X_input = X_train[0:900]
y_input = y_train[0:900]

#print (X_input.shape)
#print (y_input.shape)

X_train_data = X_input[0:630]
X_test_data = X_input[630:900]

y_train_data = y_input[0:630]
y_test_data = y_input[630:900]

# Variables
hidden_layer_1_nodes = 300
hidden_layer_2_nodes = 100
output_layer_nodes = 100
epochs = 10
classes = 2
epoch_errors = []
stddev = 0.035
learning_rate = 0.08
batch_size = 100

#print (X_train_data[0])

# TF Placeholders
# input data should be of the shape (batch_size, flatten data for one example). Also, the correct shape of y"
X = tf.placeholder(tf.float32, [None, 25 * 4], name='X')
y = tf.placeholder(tf.float32, [None, classes] name='y')

# Weights Matrices. First dimension of W1 == second dimension of X
W1 = tf.Variable(tf.truncated_normal([25 * 4, hidden_layer_1_nodes], stddev=stddev), name='W1')
W2 = tf.Variable(tf.truncated_normal([hidden_layer_1_nodes, hidden_layer_2_nodes], stddev=stddev), name='W2')
W3 = tf.Variable(tf.truncated_normal([hidden_layer_2_nodes, output_layer_nodes], stddev=stddev), name='W3')
W4 = tf.Variable(tf.truncated_normal([output_layer_nodes, classes], stddev=stddev), name='W4')

# Biases Vectors
b1 = tf.Variable(tf.truncated_normal([hidden_layer_1_nodes], stddev=stddev), name='b1')
b2 = tf.Variable(tf.truncated_normal([hidden_layer_2_nodes], stddev=stddev), name='b2')
b3 = tf.Variable(tf.truncated_normal([output_layer_nodes], stddev=stddev), name='b3')
b4 = tf.Variable(tf.truncated_normal([classes], stddev=stddev), name='b4')

# Define the Neural Network
def nn_model(X):
    input_layer     =    {'weights': W1, 'biases': b1}
    hidden_layer_1  =    {'weights': W2, 'biases': b2}
    hidden_layer_2  =    {'weights': W3, 'biases': b3}
    output_layer    =    {'weights': W4, 'biases': b4}

    input_layer_sum = tf.add(tf.matmul(X, input_layer['weights']), input_layer['biases'])
    input_layer_sum = tf.nn.relu(input_layer_sum)

    hidden_layer_1_sum = tf.add(tf.matmul(input_layer_sum, hidden_layer_1['weights']), hidden_layer_1['biases'])
    hidden_layer_1_sum = tf.nn.relu(hidden_layer_1_sum)

    hidden_layer_2_sum = tf.add(tf.matmul(hidden_layer_1_sum, hidden_layer_2['weights']), hidden_layer_2['biases'])
    hidden_layer_2_sum = tf.nn.relu(hidden_layer_2_sum)

    output_layer_sum = tf.add(tf.matmul(hidden_layer_2_sum, output_layer['weights']), output_layer['biases'])
    return output_layer_sum

# Train the Neural Network
def nn_train(X):
    pred = nn_model(X)
    pred = tf.identity(pred)
    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y))
    optimizer = tf.train.AdagradOptimizer(learning_rate=learning_rate).minimize(cost)

    with tf.Session() as sess:
        init_op = tf.global_variables_initializer()
        #saver = tf.train.Saver()
        sess.run(init_op)

        for epoch in range(epochs):
            epoch_loss = 0.0

            i = 0
            while i < len(X_train_data):
                start = i
                end = i+batch_size
                # reshape before feeding.
                batch_x = np.array(X_train_data[start:end]).reshape(batch_size, 25 * 4)
                batch_y = np.array(y_train_data[start:end]).reshape(batch_size, classes)

                _, c = sess.run([optimizer, cost], feed_dict={X: batch_x, y: batch_y})
                epoch_loss += c
                i+= batch_size

            epoch_errors.append(epoch_loss)
            print('Epoch ', epoch + 1, ' of ', epochs, ' with loss: ', epoch_loss)

        correct_result = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
        accuracy = tf.reduce_mean(tf.cast(correct_result, 'float'))
        print('Acc: ', accuracy.eval({X:X_test_data.reshape(-1, 25 * 4), y:y_test_data.reshape(-1, classes)}))

if __name__ == "__main__":
    nn_train(X)

Upvotes: 0

newcomer
newcomer

Reputation: 3

UPDATE: Sorry I posted the wrong error: @Kumar, I changed the batch size to 30 (so 21*30 = 630). It prints now the epochs, but in some weird way: 

Epoch  1  of  10  with loss:  1680690.2648780346
Epoch  2  of  10  with loss:  2382142.9208984375
Epoch  3  of  10  with loss:  4215628.857421875
Epoch  4  of  10  with loss:  9046892.295166016
Epoch  5  of  10  with loss:  23961644.453125
Epoch  6  of  10  with loss:  31733882.34375
Epoch  7  of  10  with loss:  46124696.609375
Epoch  8  of  10  with loss:  61760446.28125
Epoch  9  of  10  with loss:  89145610.59375
Epoch  10  of  10  with loss:  121249417.25

And I received a next error for:

 print('Acc: ', accuracy.eval({X:X_test_data, y:y_test_data}))

ValueError: Cannot feed value of shape (270, 25, 4) for Tensor 'X:0', which has shape '(?, 100)'

Process finished with exit code 1

Upvotes: 0

Related Questions