CODEWITHSUNDEEP
pythonpython-3.xtensorflowkeras
Arnaldo Gualberto
Arnaldo Gualberto

Reputation: 334

Tensorflow - loss not decreasing

Lately, I have been trying to replicate the results of this post, but using TensorFlow instead of Keras. However, my model loss is not converging as in the code provided. I took care to use the same parameters used by the author, even those not explicitly shown. My complete code can be seen here.

I have already tried different learning rates, optimizers, and batch sizes, but these did not affect the result very much as well.

I found a bunch of other questions related to this problem here in StackOverflow and StackExchange, but most of them had no answer at all. The questions with answers, however, did not help.

I'm using TensorFlow 1.1.0, Python 3.6 and Windows 10.

The most weird thing is that we have the same database and the same model, but just different frameworks. Thus, it was not supposed to give completely different behaviours. Does anyone have suggestions about what should I try to solve this problem, please?

Upvotes: 2

Views: 5345

Answers (1)

Ryan Stout
Ryan Stout

Reputation: 1028

I ran your code basically unmodified, but I looked at the shape of your tf_labels and logits and they're not the same. logits had shape (batch_size,1,1,1) (because you were using a 1x1 convolutional filter) and tf_labels had shape (batch_size,1). I changed your loss line to be

    loss = tf.reduce_mean(tf.squared_difference(tf.squeeze(tf_labels),
           tf.squeeze(logits)))

and the loss decreased each epoch.

Upvotes: 8

Related Questions