CODEWITHSUNDEEP
pythonpandasnumpytensorflow
bsky
bsky

Reputation: 20222

Getting same prediction with tensorflow on every example

I have the following code where I am training a neural network on the MNIST dataset. Then, with the trained network I am trying to predict the values in test_inputs.

import pandas as pd
import numpy as np
import tensorflow as tf
from math import trunc
from subprocess import check_output

def make_one_hot(m):
    result = pd.DataFrame((np.asarray(m)[:,None] == np.arange(10)).astype(int))
    return result

train_data = pd.read_csv("../input/train.csv", delimiter=',')
train_labels = make_one_hot(train_data.ix[:, 0])
train_inputs = train_data.ix[:, 1:]
test_inputs = pd.read_csv("../input/test.csv", delimiter=',')    

x = tf.placeholder(tf.float32, [None, 784])

W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
y = tf.nn.softmax(tf.matmul(x, W) + b)
y_ = tf.placeholder(tf.float32, [None, 10])
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(tf.clip_by_value(y,1e-10,1.0)), reduction_indices=[1]))
train_step = tf.train.GradientDescentOptimizer(0.05).minimize(cross_entropy)


sess = tf.InteractiveSession()
tf.global_variables_initializer().run()
for _ in range(1000):
  batch_xs = train_inputs.sample(n=100)
  batch_ys = train_labels.sample(n=100)
  sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

y = tf.nn.softmax(tf.matmul(x,W) + b)
result = sess.run(y, feed_dict={x: test_inputs})

f = open("results.csv","w+")
f.write("ImageId,Label\n")
for i in range(0, len(result)):
  x = 0
  for j in range(0, 10):
      if(result.item(i, j) == 1):
          x = j
  f.write("{},{}\n".format(i+1, x))

However, the network always predicts the digit the same digit for all examples, regardless of the inputs.

The digit itself changes, sometimes it's 1, other times6 or 7 but all the examples get the same digit for the same run.

Any idea what could be wrong with it?

EDIT:

I fixed the indentation of the last line(which was wrong) but still the result has the same label for all test examples.

Upvotes: 1

Views: 722

Answers (2)

jack6e
jack6e

Reputation: 1522

Edited in response to OP edits

I still think you are meaning to use x in your f.write iteration. Right now you are not printing to your file anything other than the value of your i iteration and your j iteration, but not actually any of your results.

Are you trying to do something like:

for i in range(0, len(result)):
    x = 0
    for j in range(0, 10):
        if(result.item(i, j) == 1):
            x = j
            f.write("{},{}\n".format(i+1, x))

Or maybe:

for i in range(0, len(result)):

    for j in range(0, 10):
        x = result.item(i, j)
        f.write("{},{}\n".format(i+1, x))

Original response

You need to fix your indentation:

for j in range(0, 10):
    if(result.item(i, j) == 1):
        x = j
    f.write("{},{}\n".format(i+1, j))

so that the f.write command falls under the for j loop. Right now you are writing the last j value, which in a range(0,10) is always 9 after the loop runs, which means every i image_id gets the label 9. Also, what are you doing with x? You first set x = 0 and then x = j based on a condition, but never actually use it.

Upvotes: 2

Mark
Mark

Reputation: 414

The initialization of the variables will not allow the network to train. Having both biases and Weights initialized as zero will lead to no training as the gradients are always zero.

Use truncated normal with mean not equal to zero (around 0.1 is good). https://www.tensorflow.org/api_docs/python/tf/truncated_normal

And the biases as ones. This will help the network to train as now the gradients won't be zero.

Upvotes: 2

Related Questions