CODEWITHSUNDEEP
pythontensorflowkerasneural-networkloss-function
F1sher
F1sher

Reputation: 7310

How to create your own loss function for tf.keras that uses additional parameter?

My need:

I would like to modify my loss function in Neural Network by adding sample weights. (I am aware that .fit method has sample_weight parameter).

My idea was to create additional input to my Neural Network with precomputed weights for each train data row like this:

# Generating mock data
train_X = np.random.randn(100, 5)
train_Y = np.random.randn(100, 1)
train_sample_weights = np.random.randn(*train_Y.shape)

# Designing loss function that uses my pre-computed weights
def example_loss(y_true, y_pred, sample_weights_):
    return K.mean(K.sqrt(K.sum(K.pow(y_pred - y_true, 2), axis=-1)), axis=0) * sample_weights_

# Two inputs for neural network, one for data, one for weights
input_tensor = Input(shape=(train_X.shape[1],))
weights_tensor = Input(shape=(train_sample_weights.shape[1],))

# Model uses only 'input_tensor'
x = Dense(100, activation="relu")(input_tensor)
out = Dense(1)(x)

# The 'weight_tensor' is inserted into example_loss() functon
loss_function = partial(example_loss, sample_weights_=weights_tensor)

# Model takes as an input both data and weights
model = Model([input_tensor, weights_tensor], out)
model.compile("Adam", loss_function)
model.fit(x=[train_X, train_sample_weights], y=train_Y, epochs=10)

My problem:

Following code works when I use Keras 2.2.4 imports to run it:

import numpy as np
from functools import partial

import keras.backend as K
from keras.layers import Input, Dense
from keras.models import Model

Following code crashes when I use tf.keras 2.2.4-tf imports to run it:

import numpy as np
from functools import partial

import tensorflow.keras.backend as K
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model

With the following error:

TypeError: example_loss() got an unexpected keyword argument 'sample_weight'

My questions:

  1. Why this happens?
  2. How could I rewrite the code so such architecture could also work on 2.2.4-tf?
  3. Proposition that works on both Keras/tf.keras frameworks is also an acceptable answer for me.

Error is easy to reproduce. Just need to copy the code and run.

Upvotes: 3

Views: 486

Answers (2)

Dr. Snoopy
Dr. Snoopy

Reputation: 56347

You can rewrite your loss like this:

# Designing loss function that uses my pre-computed weights
def example_loss(sample_weights_):
    def loss(y_true, y_pred):
        return K.mean(K.sqrt(K.sum(K.pow(y_pred - y_true, 2), axis=-1)), axis=0) * sample_weights_

As you see, here we have a function that takes the sample weights, and returns another function (the actual loss) that has the sample weights embedded into it. You can use it as:

model.compile(optimizer="adam", loss=example_loss(weights_tensor))

Upvotes: 1

Thibault Bacqueyrisses
Thibault Bacqueyrisses

Reputation: 2331

You need to define your loss like that in order to pass new parameters to it :

def custom_loss(sample_weights_):

    def example_loss(y_true, y_pred):
        return K.mean(K.sqrt(K.sum(K.pow(y_pred - y_true, 2), axis=-1)), axis=0) * sample_weights_

    return example_loss

and call it like that :

model.compile("Adam", custom_loss(weights_tensor))

Upvotes: 4

Related Questions