CODEWITHSUNDEEP
nantensorflowloss
u4lr
u4lr

Reputation: 41

Loss turns to be NAN at the first round step in my tensorflow CNN

Loss turns to be NAN at the first round step in my tensorflow CNN.

1. Network :

3 hidden layers (2 convolutional layers +1 hidden fullconnect layer) + readout layer.

2. The 3 hidden layers :

a) Weights :
W = tf.Variable(tf.truncated_normal(wt,stddev=0.1,name='wights' ))

b) Bias :
b = tf.Variable( tf.fill([W.get_shape().as_list()[-1] ],0.9),name = 'biases' )

c) Activition:
ReLu

d) Dropout:
0.6

**loss turns to be nan even if dropout is 0.0

3. Readout layyer:
softmax

4: loss function:
tf.reduce_mean(-tf.reduce_sum(_lables * tf.log(_logist), reduction_indices=[1]))

5.optimizer:
tf.train.AdamOptimizer

learning_rate = 0.0005
**loss truns to be nan even if learning_rate = 0

Upvotes: 0

Views: 1173

Answers (2)

Jeffrey Sun
Jeffrey Sun

Reputation: 8049

So far I've hit two cases where nan can result:

  • The numbers get too large (perhaps you're performing square on a number, and the result is too large)
  • Or there exists an invalid input (some functions like sqrt and log don't take negative inputs, so they would return nan)

Upvotes: 1

Sung Kim
Sung Kim

Reputation: 8536

Since we don't have the entire source code, it's hard to see the problem. However, you may try to use 'tf.nn.softmax_cross_entropy_with_logits' in your cost function. For example:

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(predictions, labels))

You can find an entire code example using 'tf.nn.softmax_cross_entropy_with_logits' at https://github.com/nlintz/TensorFlow-Tutorials.

Upvotes: 1

Related Questions