CODEWITHSUNDEEP
neural-networktensorflow
Tamim Addari
Tamim Addari

Reputation: 7831

Simple tensorflow neural network not increasing accuracy or decreasing loss?

I have the following network for training,

graph = tf.Graph()
with graph.as_default():

    tf_train_dataset = tf.constant(X_train)
    tf_train_labels = tf.constant(y_train)
    tf_valid_dataset = tf.constant(X_test)

    weights = tf.Variable(tf.truncated_normal([X_train.shape[1], 1]))

    biases = tf.Variable(tf.zeros([num_labels]))
    logits = tf.nn.softmax(tf.matmul(tf_train_dataset, weights) + biases)

    loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits, tf_train_labels))
    optimizer = tf.train.GradientDescentOptimizer(0.5).minimize(loss)
    train_prediction = tf.nn.softmax(logits)
    valid_prediction = tf.nn.softmax(tf.matmul(tf_valid_dataset, weights) + biases)

and I ran it as follows,

num_steps = 10

with tf.Session(graph=graph) as session: 
    tf.initialize_all_variables().run()
    print('Initialized')
    for step in range(num_steps):
        _, l, predictions = session.run([optimizer, loss, train_prediction])
        print("Loss: ",l)
        print('Training accuracy: %.1f' % sklearn.metrics.accuracy_score(predictions.flatten(), y_train.flatten()))

But it outputes as follows 

Initialized
Loss:  0.0
Training accuracy: 0.5
Loss:  0.0
Training accuracy: 0.5

The shape of X_train is (213403, 25) and y_train is (213403, 1) to cope up with the logits. I didn't encode the the labels as one hot because there are only two classes , either 1 or 0. I also tried with quadratic loss function and it was still the same, and same thing happened, the loss function doesn't decrease at all. I am sensing a syntactical mistake here but I am clueless.

Upvotes: 1

Views: 1338

Answers (1)

Pramod Patil
Pramod Patil

Reputation: 827

Your are passing a labels as a single column(without encoding). Model is unable to get labels as factor type. So it considers your labels as continuous value.

Loss: 0.0 means loss is zero. That means your model is perfectly fit. This is happening because your labels are continuous(regression function) and you are using softmax_cross_entropy_with_logits loss function.

Try passing one hot encoding of labels and check.

Upvotes: 2

Related Questions