CODEWITHSUNDEEP
pythontensorflowkerasconv-neural-network
Denisa Ionascu
Denisa Ionascu

Reputation: 33

What should be the output of the custom loss function in Keras?

I am trying to build a custom loss function in Keras, but I am confused about the way it works. I am training the network on batches, and I am not sure if the output of the loss function should be an array with the same dimension as the batch or just a scalar.

Upvotes: 2

Views: 2241

Answers (2)

Vlad
Vlad

Reputation: 8585

The loss usually reduced over all dimensions of the mini-batch. If you don't apply reduction it would be performed implicitly (try removing tf.reduce_mean in custom_loss_function() and return just res). For example:

import tensorflow as tf
import numpy as np

def custom_cross_entropy(y_true, y_pred):
    res = -y_true*tf.math.log(tf.nn.softmax(y_pred))
    return tf.reduce_mean(res, axis=None)

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(2, activation=None))

model.compile(optimizer=tf.keras.optimizers.SGD(0.01),
              loss=[custom_cross_entropy],
              metrics=['accuracy'])

y_train = np.array([[1, 0], [0, 1]])
x_train = np.random.normal(size=(2, 2))

model.fit(x_train, y_train, epochs=2)

# Epoch 1/2
# 2/2 [==============================] - 0s 13ms/sample - loss: 0.2689 - accuracy: 1.0000
# Epoch 2/2
# 2/2 [==============================] - 0s 2ms/sample - loss: 0.2686 - accuracy: 1.0000

Upvotes: 1

Deusy94
Deusy94

Reputation: 757

As described in the documentation, keras loss, You can pass a function that returns a scalar for each data-point and takes the two arguments: y_true (True labels) and y_pred (Predictions).

Keras execute the mean over sample inside batch, so the output should just be a single scalar.

Upvotes: 3

Related Questions