CODEWITHSUNDEEP
pythontensorflowconv-neural-networkpredictionloss-function
Lucci
Lucci

Reputation: 3

How to get prediction when computing loss function in convolutional neural network (tensorflow)?

I built a convolutional neural network with tensorflow by following these steps: https://www.tensorflow.org/tutorials/estimators/cnn

I want to compute the loss with my own loss function and therefore need to get the predicted propabilities of each class in each training step. From the Tensorflow tutorial I know that I can get these propabilities with "tf.nn.softmax(logits)", however this returns a tensor and I don't know how to extract the actual propabilities from this tensor. Can anyone please tell me how I can get these propabilities, so I can compute my loss function?

Upvotes: 0

Views: 331

Answers (1)

gorjan
gorjan

Reputation: 5555

This is how you compute the softmax and get the probabilities afterwards:

# Probabities for each element in the batch for each class.
softmax = tf.nn.softmax(logits, axis=1)
# For each element in the batch return the element that has the maximal probability
predictions = tf.argmax(softmax, axis=1)

However, please note that you don't need the predictions in order to compute the loss function, you need the actuall probabilities. In case you want to compute other metrics then you can use the predictions (metrics such as accuracy, precision, recall and ect..). The softmax Tensor, contains the actual probabilities for each of your classes. For example, assuming that you have 2 elements in a batch, and you are trying to predict one out of three classes, the softmax will give you the following:

# Logits with random numbers
logits = np.array([[24, 23, 50], [50, 30, 32]], dtype=np.float32)
tf.nn.softmax(logits, axis=1)
# The softmax returns
# [[5.1090889e-12 1.8795289e-12 1.0000000e+00]
#  [1.0000000e+00 2.0611537e-09 1.5229979e-08]]
# If we sum the probabilites for each batch they should sum up to one
tf.reduce_sum(softmax, axis=1)
# [1. 1.]

Based on how you imagine your loss function to be this should be correct:

first_second = tf.nn.l2_loss(softmax[0] - softmax[1])
first_third = tf.nn.l2_loss(softmax[0] - softmax[2])
divide_and_add_m = tf.divide(first_second, first_third) + m
loss = tf.maximum(0.0, 1 - tf.reduce_sum(divide_and_add_m))

Upvotes: 0

Related Questions