CODEWITHSUNDEEP
pythontensorflowmachine-learningmnist
Mohamed Mostafa
Mohamed Mostafa

Reputation: 620

simple MLP from scratch using tensorflow

I am trying to implement MLP in tensorflow from scratch and test it on MNIST dataset. This is my code:

import tensorflow.compat.v1 as tf
from tensorflow.compat.v1.keras.losses import categorical_crossentropy
tf.disable_v2_behavior()

image_tensor = tf.placeholder(tf.float32 , shape=(None , 784))
label_tensor = tf.placeholder(tf.float32 , shape=(None , 10))

# Model architecture
# --> Layer 1
w1 = tf.Variable(tf.random_uniform([784 , 128])) # weights
b1 = tf.Variable(tf.zeros([128])) # bias
a1 = tf.matmul(image_tensor , w1) + b1
h1 = tf.nn.relu(a1)
# --> Layer 2
w2 = tf.Variable(tf.random_uniform([128 , 128]))
b2 = tf.zeros([128])
a2 = tf.matmul(h1 , w2) + b2
h2 = tf.nn.relu(a2)
# --> output layer
w3 = tf.Variable(tf.random_uniform([128 , 10]))
b3 = tf.zeros([10])
a3 = tf.matmul(h2 , w3) + b3
predicted_tensor = tf.nn.softmax(a3) 

loss = tf.reduce_mean(categorical_crossentropy(label_tensor , predicted_tensor))

opt = tf.train.GradientDescentOptimizer(0.01) 
training_step = opt.minimize(loss)

with tf.Session() as sess:
    init_op = tf.global_variables_initializer()
    sess.run(init_op)   
    epochs = 50
    batch  = 100
    iterations = len(training_images) // batch

    for j in range(epochs):
        start = 0
        end = batch
        for i in range(iterations):
            image_batch = np.array(training_images[start : end])
            label_batch = np.array(training_labels[start : end])

            start = batch + 1
            end = start + batch
            _ , loss = sess.run(training_step  , feed_dict = {
                image_tensor : image_batch,
                label_tensor : label_batch
                })

but when I try to run this code I get this error message:

File "MNIST3.py", line 97, in <module>
    main()
  File "MNIST3.py", line 88, in main
    label_tensor : label_batch
TypeError: 'NoneType' object is not iterable

Although when I try to print the first 10 samples from the label_batch:

print(training_labels[0 : 10])

This will be the output:

[[1 0 0 0 0 0 0 0 0 0]
 [1 0 0 0 0 0 0 0 0 0]
 [1 0 0 0 0 0 0 0 0 0]
 [1 0 0 0 0 0 0 0 0 0]
 [1 0 0 0 0 0 0 0 0 0]
 [1 0 0 0 0 0 0 0 0 0]
 [1 0 0 0 0 0 0 0 0 0]
 [1 0 0 0 0 0 0 0 0 0]
 [1 0 0 0 0 0 0 0 0 0]
 [1 0 0 0 0 0 0 0 0 0]]

And when I try to print the shape of the dataset:

print(training_images.shape)
print(training_labels.shape)

This is the output:

(10000, 784)
(10000, 10)

What am I missing here?

Upvotes: 0

Views: 183

Answers (1)

Addy
Addy

Reputation: 1450

You're misunderstanding the error message (Python can be quite misleading in this, we've all gotten stuck on errors like this one more times than we'd like to admit...). Even though it displays the label_tensor : label_batch line in your error, it's actually talking about the whole session.run() call.

And the reason, why you're seeing this error is because you're expecting a tuple to be returned by the call but you provide only a single tensor to be computed by TensorFlow.

sess.run(training_step, feed_dict=...) will return None because the op training_step is not supposed to return anything, calling it you just perform one optimization step.

To get the desired result, change the code to:

_ , loss_result = sess.run([training_step, loss], 
                           feed_dict={
                               image_tensor : image_batch,
                               label_tensor : label_batch
                           })

This way TensorFlow will evaluate those 2 ops, first will return None (as you're getting already) and the second one will compute the value of your loss function for given batch.

(Notice that you have to rename the loss variable on the left side because if you don't you will replace the loss op and the next call would probably raise exception or worse, just give wrong results)

Upvotes: 1

Related Questions